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Executive  Summary 

The  characterization  of  uncertainty  in  the  estimate  of  the  state  of  a  resident  space  object  is  fundamental 
to  many  space  surveillance  tasks  including  data  association,  uncorrelated  track  (UCT)  resolution,  catalog 
maintenance,  sensor  tasking  and  scheduling  as  well  as  space  situational  awareness  (SSA)  missions  such  as 
conjunction  assessments  and  maneuver  detection.  Generally,  uncertainties  arc  classified  as  either  aleatoric, 
epistemic  or  a  mixture  of  both.  Aleatoric  uncertainty  is  the  natural  randomness  or  physical  variability 
present  in  the  system  or  its  environment  and  is  thus  statistical  in  nature.  In  contrast,  epistemic  uncertainty  is 
uncertainty  that  is  due  to  limited  data  or  knowledge  as  discussed  in  Chapter  2.  While  aleatoric  uncertainty 
is  the  focus  of  this  report,  epistemic  uncertainties  certainly  deserve  more  attention  by  the  astrodynamics 
community. 

With  respect  to  some  terms,  covariance  realism  means  that  the  uncertainty  in  the  state  of  an  object  can 
be  represented  as  a  Gaussian  random  variable  and  that  the  estimated  mean  and  covariance  of  said  Gaussian 
are  the  true  mean  and  true  covariance,  respectively.  Since  the  underlying  dynamical  processes  are  not  always 
linear  nor  Gaussian,  one  may  generalize  covariance  realism  to  uncertainty  realism  described  by  a  potentially 
non-Gaussian  probability  density  function.  Uncertainty  realism  requires  that  all  cumulants  (beyond  a  state 
and  covariance)  be  properly  characterized.  The  relationship  between  covariance  realism  and  uncertainty 
realism  is  that  the  former  is  a  necessary  but  not  a  sufficient  condition  for  achieving  the  latter.  The  two 
definitions  coincide  if  the  process  is  Gaussian.  These  terms  are  used  throughout  the  report. 

The  achievement  of  covariance  or  uncertainty  realism  is  a  challenging  problem  due  to  the  complex  and 
numerous  sources  of  uncertainty.  To  achieve  a  proper  characterization  of  uncertainty,  one  must  account  for 
the  uncertainty  sources  in  the  system  and  roll  these  up  into  the  uncertainty  in  the  estimate  at  each  needed 
time.  Generic  sources  of  uncertainties  for  point  objects  include  the  following: 

1.  Structural  uncertainty  or  model  bias  in  the  model  dynamics; 

2.  Uncertain  parameters  found  in  the  model  dynamics  (including  space  environment)  and  in  the  mea¬ 
surement  equation  relating  the  dynamics  to  the  sensor  measurements; 

3.  Sensor  level  errors  including  measurement  noise  and  sensor  and  navigation  biases; 

4.  Inverse  uncertainty  quantification  including  the  statistical  orbit  determination  and  bias  estimation 
uncertainty; 

5.  Propagation  of  uncertainty, 

6.  Algorithmic  uncertainty  or  numerical  uncertainty  that  comes  from  numerical  errors  and  numerical 
approximations  in  a  computer  model; 

7.  Cross-tag  or  misassociation  uncertainty, 

8.  Hardware  and  software  faults/errors. 

Most  of  these  errors  for  point  objects  are  pursued  in  the  subsequent  chapters.  Additional  sources  of  un¬ 
certainty  occur  for  medium  to  large  objects  called  extended  body  uncertainties.  For  example,  an  extended 
body  covering  several  pixels  may  have  an  overly  optimistic  (too  small)  covariance  if  the  uncertainty  of  the 
estimated  state  only  covers  the  centroid  of  the  body. 

The  goal  of  correctly  characterizing  or  quantifying  uncertainty  is  not  unique  to  astrodynamics.  Indeed, 
the  currently  active  field  of  uncertainty  quantification  deals  with  the  same  problem  in  the  many  other  areas 
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of  engineering  and  science.  Thus,  brief  comments  on  the  field  of  uncertainty  quantification  and  definitions 
are  provided  in  Chapter  2. 

As  stated  above,  the  correct  characterization  the  uncertainty  in  the  state  of  each  object  is  fundamental  to 
many  space  surveillance  and  space  situational  awareness  missions.  Four  examples  arc  provided  in  Chapter  3 
to  demonstrate  the  importance  of  covariance  and  uncertainty  realism,  namely 

1.  computation  of  the  probability  of  collision  for  conjunction  assessment, 

2.  data  or  track  association/correlation, 

3.  maneuver  detection, 

4.  sensor  tasking  and  scheduling. 

The  primary  goal  of  this  Chapter  3  is  to  explain  the  impact  of  having  an  unrealistic  covariance  (or,  more 
generally,  an  unrealistic  probability  density  function)  that  characterizes  the  state  of  a  space  object. 

Chapter  4  presents  a  macroscopic  view  of  the  uncertainties  in  generic  reference  systems,  time  systems, 
and  astrodynamic  force  models  used  as  a  reference  for  discussing  the  uncertainties  in  these  models.  In 
addition,  an  attempt  is  made  to  delineate  and  quantify  the  structural  and  parametric  uncertainties.  Rec¬ 
ommendations  for  reducing  and  properly  characterizing  these  uncertainties  are  given  in  Section  4.3.  It  is 
anticipated  that  a  future  report  on  New  Methods  of  Reentry  Modeling  will  considerably  expand  the  topic  of 
uncertainty  quantification  of  reentering  objects. 

The  importance  of  a  complete  understanding  and  correct  characterization  of  sensor  errors  and  their 
temporal  behavior  is  required  to  achieve  covariance  and  uncertainty  realism  of  a  sensor  report.  This  is 
perhaps  one  of  the  most  important  topics  that  has  received  little  attention  in  the  literature  and  yet  could  offer 
the  biggest  payoff  to  improving  uncertainty  realism  in  SSA.  This  topic  is  discussed  in  Chapter  5.  First,  one 
has  the  natural  randomness  of  the  intrinsic  sensor  system  in  which  observations  are  made.  Next,  one  has  to 
consider  a  bias  model  for  sensor  calibration  that  includes  atmospheric,  sensor,  and  navigation  biases.  After 
these  are  estimated,  one  must  address  the  uncertainties  in  the  estimates.  What  is  often  reported  in  a  sensor 
report  is  not  an  intrinsic  observable,  but  instead  a  filtered  state  transformed  into  yet  another  coordinate 
system.  Thus,  a  short  burst  of  measurements  over  a  few  seconds  or  minutes  may  be  highly  correlated.  The 
challenge  then  is  to  roll  up  all  of  these  uncertainties  into  a  sensor  report  or  reports  and  to  obtain  realistic 
uncertainties. 

Having  presented  the  uncertainties  in  sensor  level  processing,  dynamical  models  including  atmospheric 
models,  Chapter  6  surveys  some  of  the  formulations  and  methods  for  the  inverse  problem  of  extracting  a 
state,  e.g.,  position  and  velocity,  from  the  sensor  reports.  In  particular,  this  chapter  reviews  many  of  the 
nonlinear  estimation  techniques  including  filtering,  smoothing,  prediction,  and  batch  estimation  that  are 
currently  used  in  the  astrodynamics  community.  The  focus  is  on  the  estimation  techniques  that  yield  the 
state  and  its  uncertainty  at  the  time  of  the  last  measurement,  i.e.,  epoch,  as  opposed  to  the  propagation  of 
uncertainty  after  the  last  sensor  report.  Included  in  this  chapter  is  a  discussion  of  the  different  process  noise 
or  stochastic  acceleration  models  often  used  by  the  astrodynamics  community.  Model  bias  is  also  treated  as 
a  control  problem  that  is  discussed  in  the  earlier  Section  3.3. 

While  the  previous  chapter  examined  estimation  methods  through  epoch,  Chapter  7  addresses  the  propa¬ 
gation  of  uncertainty  in  support  of  the  SSA  missions  discussed  in  Chapter  3  assuming  that  the  quantification 
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of  uncertainty  at  epoch  is  correct.  There  are  several  considerations  that  affect  the  quality  of  the  representa¬ 
tion  of  uncertainty  and  the  length  of  time  it  remains  valid.  First,  the  coordinate  system  in  which  uncertainty 
is  represented  and  propagated  has  a  significant  impact.  Generally  this  means  a  Cartesian  versus  an  orbital 
element  space  as  discussed  in  Section  7.2.  Regardless  of  which  coordinate  system  is  used,  each  of  the 
aforementioned  mission  areas  may  require  the  representation  in  a  different  coordinate  system  than  that  in 
which  uncertainty  is  propagated.  Thus,  one  must  address  the  faithful  representation  in  the  needed  coor¬ 
dinate  system.  Research  into  the  topic  of  “uncertainty  propagation”  by  the  astrodynamics  community  is 
broad  and  quite  excellent.  Thus,  the  goal  of  this  chapter  is  to  survey  most  of  these  methods  and  to  make 
recommendations  on  approaches  that  might  be  used  to  evaluate  the  different  methods  for  different  purposes. 

The  scope  of  this  report  is  somewhat  limited.  As  indicated  above,  the  uncertainties  treated  are  those 
of  the  aleatoric  as  opposed  to  epistemic  uncertainties.  Secondly,  no  systematic  attempt  has  been  made 
to  evaluate  any  existing  or  proposed  operational  system  nor  make  recommendations  to  improve  uncertainty 
realism  for  such  a  system.  This  was  necessary  given  the  Distribution-A  requirements  on  the  report;  however, 
we  recommend  that  such  a  study  be  undertaken  as  part  of  a  separate  effort.  In  keeping  with  this  requirement, 
the  intended  audience  is  a  general  one.  While  there  are  some  recommendations  on  specific  algorithms 
throughout  the  report,  no  systematic  assessment  of  the  different  algorithms  has  been  undertaken.  Instead 
of  the  committee  making  an  assessment  of  the  different  algorithms  and  approaches,  the  recommendation  is 
to  set  up  an  environment  whereby  one  can  evaluate  the  different  algorithms.  Toward  this  end,  metrics  are 
proposed  in  Chapter  8  and  an  initial  list  of  benchmark  test  cases  are  proposed  in  Chapter  9  for  a  subset  of 
the  problems  considered  in  this  report. 

The  concluding  Chapter  10  briefly  reviews  the  above  chapters  from  the  viewpoint  of  the  aforementioned 
generic  uncertainties  and  presents  recommended  “Next  steps”  in  Section  10.5. 
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Chapter  1 


Introduction 


Space  situational  awareness  (SSA)  encompasses  intelligence,  surveillance  of  all  space  objects,  and  the  pre¬ 
diction  of  space  events,  possible  collisions,  threats,  and  activities  [1].  Fundamental  to  SSA  are  conjunction 
analysis  (probability  of  collision),  sensor  tasking  and  scheduling,  data/track  association  for  uncorrelated 
track  (UCT)  resolution  and  catalog  maintenance,  and  anomaly  (e.g.,  maneuver,  change)  detection.  Common 
amongst  these  missions  is  the  requirement  of  a  proper  characterization  of  the  uncertainty  and  errors  in  the 
estimation  of  each  resident  space  object  (RSO),  which  is  called  covariance  or  uncertainty  realism. 

The  process  of  achieving  covariance  or  uncertainty  realism  lies  in  the  realm  of  space  surveillance,  which 
is  that  component  of  SSA  focused  on  the  detection  of  space  objects  and  on  the  use  of  multi-source  data  to  de¬ 
tect,  track,  identify  and  characterize  space  objects.  Space  surveillance  presents  some  unique  and  formidable 
challenges  not  found  in  other  tracking  environments.  In  contrast  to  air,  missile,  or  ground  tracking,  the  space 
surveillance  environment  is  data-starved',  however  the  dynamic  models  have  been  and  continue  to  be  well 
developed.  Typical  resident  space  object  (RSO)  tracking  problems  can  require  the  long  term  propagation  of 
state  probability  density  functions  (PDFs),  often  on  the  order  of  several  orbital  periods,  using  high  fidelity 
dynamical  models  in  the  absence  of  measurement  or  track  updates.  Even  if  the  uncertainty  in  the  state  of 
an  object  is  characterized  as  a  Gaussian  (represented  by  an  ellipsoidal  covariance)  at  some  point  in  time, 
the  state  will  inevitably  become  non-Gaussian  (non-ellipsoidal)  if  propagated  for  a  sufficiently  long  time 
span  under  nonlinear  dynamics  (i.e.,  gravity,  drag,  solar  radiation,  third-body  perturbations,  etc.).  Conse¬ 
quently,  the  term  covariance  realism  used  in  data-rich  tracking  environments  and  referring  to  the  accurate 
and  truthful  representation  of  the  errors  of  an  estimate  as  a  Gaussian  random  variable  is  only  a  prerequisite 
to  the  more  general  notion  of  uncertainty  realism  in  which  these  errors  are  characterized  by  a  more  general 
probability  density  function  (PDF). 

Without  the  proper  representation  of  uncertainty,  false  conclusions  can  arise:  non-predicted  collisions 
do  occur,  RSOs  declared  to  be  UCTs  do  in  fact  correlate,  sensor  tasking  and  scheduling  algorithms  lead  to 
inefficient  use  of  sensor  resources,  and  maneuvers  go  undetected.  Compounding  each  of  these  issues  is  the 
expected  substantial  growth  in  the  number  of  observed  RSOs.  With  improved  sensors  such  as  the  newly 
funded  Space  Fence  (S-Band  radar)  and  improved  electro-optical  (EO)  sensors  such  as  Pan-STARRS,  the 
number  of  observed  RSOs  will  increase  dramatically.  Debris  fields  left  by  the  collision  of  satellites  such 
as  the  recent  Iridium-Cosmos  collision  or  destruction  of  satellites  as  in  the  Chinese  ASAT  test  is  a  major 


1 


1.1.  GENERIC  SOURCES  OF  ERRORS  AND  UNCERTAINTY  IN  SSA'H  AFTER  1 .  INTRODUCTION 


and  growing  threat  to  existing  satellites.  Thus,  for  space  protection  and  space  situational  assessment,  an 
automated  system  is  needed  to  track  and  identify  the  ever  increasing  number  of  objects  in  space  and  to 
support  the  aforementioned  mission  areas.  Such  an  automated  system  will  inevitably  be  statistics-based 
approach  to  surveillance.  This  in  turn  requires  the  correct  accounting  and  faithful  characterization  of  the 
different  uncertainties  and  errors  throughout  the  Space  Surveillance  Network  (SSN),  the  orbit  estimation 
algorithms  in  the  Command  and  Control  (C2)  center  and  a  roll-up  of  these  into  that  of  the  state  of  each 
RSO. 

Thus,  the  goals  of  this  report  arc  to  identify  the  uncertainties  and  errors  throughout  the  SSN,  to  survey 
some  of  the  associated  work  in  the  astrodynamics  community  for  treating  these  uncertainties,  and  to  make 
recommendations  for  improving  such  or  at  least  to  set  up  a  collaborative  environment  that  can  lead  to  such 
improvements.  Not  all  uncertainties  or  errors  contribute  equally  to  the  final  rollup,  so  a  subgoal  is  to  identify 
the  more  dominant  ones.  Ideally  one  would  end  up  with  a  list  of  the  uncertainties  that  dominate  the  final 
rollup,  so  they  could  methodically  be  reduced. 

Generic  sources  of  errors  and  uncertainty  in  other  surveillance  systems  have  been  identified  by  Drum¬ 
mond  [75]  and  are  expanded  to  the  same  for  space  surveillance  in  Section  1.1.  Topics  not  addressed  in 
this  report  are  outlined  in  Section  1.2;  however,  they  should  be  investigated  in  any  future  work.  Finally,  an 
outline  of  the  report  is  provided  in  Section  1.3. 

1.1  Generic  Sources  of  Errors  and  Uncertainty  in  SSA 

Again,  uncertainty  quantification  in  SSA  deals  with  the  quantification  of  uncertainty  and  the  achievement 
of  uncertainty  realism  in  the  SSN  at  each  stage  of  the  processing  with  the  objective  being  to  roll-up  this 
uncertainty  into  the  state  of  each  resident  space  object.  Motivated  by  similar  uncertainties  considered  in  the 
field  of  uncertainty  quantification,  here  are  some  generic  uncertainties  for  point  objects. 

1.  Structural  uncertainty  is  also  known  as  model  inadequacy,  model  bias,  or  model  discrepancy,  which 
comes  from  the  lack  of  knowledge  of  the  underlying  astrodynamical  forces  acting  on  a  RSO  as  well 
as  that  of  maneuvers. 

2.  Uncertain  parameters  are  found  in  the  model  dynamics  (including  space  environment)  and  in  the 
measurement  equation  relating  the  dynamics  to  the  sensor  measurements.  Included  in  the  latter  are  the 
sensor,  navigation,  and  time  biases  as  well  as  the  corresponding  residual  biases  and  bias  drift  between 
sensor  calibrations. 

3.  Sensor  measurement  noise,  sometimes  called  experimental  error  or  observation  error,  as  found  in 
the  measurement  equation,  is  generally  assumed  to  be  Gaussian  and  white;  however,  it  can  be  non- 
Gaussian  and  correlated.  Sensor  measurements  can  be  in  the  form  of  the  actual  or  intrinsic  mea¬ 
surements  of  the  sensor  or  pseudo  or  derived  measurements  such  as  sensor  tracks.  In  addition  to  the 
random  sensor  measurement  noise,  one  also  treats  the  residual  bias  noise  either  by  a  consider  analysis 
or  a  Schmidt- Kalman  filter. 

4.  Inverse  uncertainty  quantification  includes  the  statistical  orbit  determination  and  bias  estimation 
uncertainty  in  that  both  may  be  considered  to  be  inverse  problems.  The  estimation  of  the  state  of 
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an  object  and  its  uncertainty  given  the  sensor  reports  and  astrodynamics  models  is  critical  to  the 
propagation  of  uncertainty.  Also  included  in  this  subject  is  the  estimation  of  the  sensor,  navigation, 
or  time  biases  given  truth  objects  or  fuzzy  truth  objects  (e.g.,  high  accuracy  orbits)  and  the  statistical 
characterization  of  the  uncertainty  in  these  estimates.  (An  inverse  problem  is  a  general  framework 
that  is  used  to  convert  observed  measurements  into  information  about  a  physical  object  or  system.  A 
bias  is  a  systematic  error  that  does  not  average  out.) 

5.  Propagation  of  uncertainty  refers  to  the  propagation  of  uncertainty  in  the  state  of  an  object  and 
its  uncertainty  through  nonlinear  dynamics  or  to  the  transformation  through  nonlinear  functions.  In¬ 
cluded  in  this  problem  is  that  of  predicting  space  weather  into  the  future. 

6.  Algorithmic  uncertainty  or  numerical  uncertainty  comes  from  numerical  errors  and  numerical  ap¬ 
proximations  in  a  computer  model.  An  example  is  the  truncation  error  in  the  orbital  propagator,  which 
can  be  mitigated  by  adjusting  the  numerical  error  tolerance  so  that  the  magnitude  of  the  numerical  er¬ 
ror  is  below  that  of  the  uncertainty  due  to  the  physics  or  uncertainty  in  the  state  of  an  object.  Another 
example  is  that  of  orbit  determination  in  the  presence  of  ill-conditioning. 

7.  Cross-tag  or  misassociation  uncertainty  is  discrete  in  nature  and  is  generally  characterized  through 
the  computation  of  the  probability  of  association  [166,  205].  Cross-tags  can  degrade  the  uncertainty 
in  the  state  of  an  object  and  are  especially  important  for  closely  spaced  objects  such  as  GEO  clusters, 
LEO  breakups,  and  tethered  satellites. 

8.  Hardware  and  software  faults/errors  are  yet  another  source  of  uncertainty  introduced  into  the  pro¬ 
cess. 

9.  While  the  above  sources  are  valid  for  point  objects,  there  are  additional  sources  of  uncertainty  for 
medium  to  large  objects  called  extended  body  uncertainties.  For  example,  an  extended  body  cov¬ 
ering  several  pixels  may  have  an  overly  optimistic  (too  small)  covariance  if  the  uncertainty  of  the 
estimated  state  only  covers  the  centroid  of  the  body.  One  must  also  address  the  extent  of  the  body, 
possibly  through  the  use  of  feature  data  such  as  multi-band  photometry  and  radiometry.  Radar,  on  the 
other  hand,  may  receive  reflections  from  multiple  point  scatterers  and  the  centroided  uncertainty  may 
not  cover  the  extent  of  the  body.  (We  note  that  the  radar  cross  section  (RCS)  of  an  extended  object  is 
not  well  defined.  Each  point  scatterer  may  have  a  different  RCS.  Thus  the  “object  RCS”  could  be  the 
mean  or  the  maximum  of  the  point  scatterers.) 

1.2  Focus  of  the  Report  and  Topics  Not  Addressed 

As  indicated  above,  the  uncertainties  treated  are  those  of  the  aleatoric  as  opposed  to  epistemic  uncertain¬ 
ties.  While  this  report  surveys  many  of  the  uncertainties  in  the  space  surveillance  network  as  well  as  other 
contributing  sensor  inputs  in  a  generic  sense,  it  does  not  provide  a  comprehensive  review  in  addressing 
and  identifying  the  current  (or  legacy)  operational  system  and  its  deficiencies  with  respect  to  the  expected 
requirements  (current  or  future).  This  was  necessary  given  the  Distribution-A  limitations  of  this  report;  how¬ 
ever,  we  recommend  that  such  a  study  be  undertaken  as  part  of  a  separate  effort.  (Chapter  5  on  sensor  level 
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processing  does  make  some  generic  recommendations  along  these  lines.)  In  keeping  with  this  requirement, 
the  intended  audience  is  general.  While  there  are  some  recommendations  on  specific  algorithms  throughout 
the  report,  no  systematic  assessment  of  the  different  algorithms  has  been  undertaken.  Instead  of  trying  to 
solve  the  problem,  the  approach  has  been  to  set  up  an  environment  where  one  can  evaluate  the  different 
algorithms.  Toward  this  end,  metrics  are  proposed  in  Chapter  8  and  an  initial  list  of  benchmark  test  cases 
proposed  in  Chapter  9  for  a  subset  of  the  problems  considered  in  this  report. 

While  this  report  does  address  many  of  the  above  sources  of  uncertainty,  it  does  not  address  all  of  those 
listed  in  Section  1.1.  Here  is  such  a  list. 

•  While  the  subject  of  algorithms  is  briefly  discussed  throughout  the  different  sections,  there  is  no  in- 
depth  treatment  of  such  algorithms.  In  particular,  there  is  no  comparison  between  filter/smoothing 
and  batch  processing  and  the  corresponding  algorithms. 

•  No  attempt  is  made  to  address  extended  body  uncertainties. 

•  All  of  the  uncertainties  and  errors  treated  in  this  report  are  of  the  aleatoric  type. 

•  No  attempt  is  made  to  address  cross-tag  or  misassociation  uncertainty,  which  is  treated  to  some  extent 
in  Drummond  [77]. 

•  Metrics  and  test  cases  for  the  estimate  of  the  state  and  its  uncertainties  at  epoch  have  not  been  specif¬ 
ically  addressed. 

•  The  report  does  not  provide  an  estimate  of  the  expected  total  level  of  effort  in  completing  the  goals 
and  neither  addressing  potential  issues  or  impact  on  system  interoperability. 


1.3  Organization  of  the  Report 

The  report  is  organized  in  the  following  manner. 

•  Chapter  2  presents  some  of  the  required  definitions  and  some  of  the  salient  elements  of  the  related 
field  of  uncertainty  quantification. 

•  Chapter  3  presents  four  mission  areas  (conjunction  assessments,  data  association,  maneuver  detection, 
and  sensor  tasking  and  scheduling)  that  require  a  proper  characterization  of  errors  and  uncertainties 
in  the  state  of  a  resident  space  object. 

•  Chapter  4  surveys  some  of  the  structural  and  parametric  uncertainties  in  the  astrodynamics  equations 
of  motion  including  the  related  space  environment. 

•  Chapter  5  presents  an  overview  of  the  issues  in  sensor  level  processing  that  results  in  the  sensor  reports 
processed  by  statistical  orbit  determination.  This  chapter  also  overviews  the  sensor,  navigation,  and 
time  biases  that  are  treated  as  parametric  uncertainties  in  Section  6.2. 

•  Chapter  6  presents  a  brief  overview  of  some  of  the  methods  used  by  the  astrodynamics  community  to 
estimate  the  state  of  each  object  and  its  uncertainty.  In  particular,  parametric  structural  uncertainty  are 
discussed  in  Section  6.2.  Structural  uncertainty  due  to  model  bias  or  inadequacy  is  further  discussed 
in  Sections  6.6  and  6.7  as  well  as  the  earlier  Section  3.3 
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•  Chapter  7  surveys  some  of  the  different  methods  being  used  to  propagate  uncertainty  forward  in  time 
in  support  of  the  aforementioned  mission  areas. 

•  Chapter  8  proposes  new  metrics  to  assess  the  performance  of  different  algorithms,  especially  for 
propagating  uncertainty. 

•  Chapter  9  provides  some  sample  test  cases  that  will  need  to  be  augmented  by  the  astrodynamics 
community. 

•  Chapter  10  provides  a  summary  and  some  recommendations  for  the  “next  steps”  as  enhancements  to 
this  report. 
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Chapter  2 


Uncertainty  Quantification 


The  problem  of  characterizing  uncertainty  and  errors  is  an  active  area  of  research  called  uncertainty  quan¬ 
tification  ( UQ ),  as  are  the  associated  subjects  of  verification,  validation,  sensitivity  analysis,  and  the  quan¬ 
tification  of  margins  and  uncertainties  (QMU).  In  particular,  there  are  many  fine  lectures,  articles,  and  books 
on  this  subject.  The  report  by  the  National  Academy  of  Sciences  [54],  a  popular  internet  reference  00  [287] 
and  Stanford  University’s  Uncertainty  Quantification  website  [136]  provide  an  assessment  of  the  state-of-art 
and  provide  many  such  references  as  do  the  book  by  Smith  [246]  and  the  review  article  by  Owhadi,  Scovel, 
Sullivan,  McKerns,  and  Ortiz  [215].  In  addition,  there  are  at  least  two  journals  that  focus  entirely  on  uncer¬ 
tainty  quantification,  namely,  the  ASME  Journal  of  Verification,  Validation  and  Uncertainty  Quantification 
and  the  SIAM/ASA  Journal  on  Uncertainty  Quantification.  This  collection  of  works  provide  some  guidance 
about  questions  being  posed,  methods  being  developed,  and  answers  to  fundamental  questions  in  quantify¬ 
ing  uncertainty  in  the  state  of  resident  space  objects.  The  purpose  of  this  chapter  then  is  to  define  some  of 
the  relevant  terms  and  explain  some  of  the  approaches. 

2.1  Some  Definitions 

Following  the  AAS  inspired  definitions  of  uncertainty  and  errors,  the  following  definitions  [136]  will  be 
used  in  the  report.  “Define  errors  as  associated  to  the  translation  of  a  mathematical  formulation  into  a 
numerical  algorithm  (and  a  computational  code).  Examples  are  round-off  errors,  limited  convergence  of 
certain  iterative  algorithms  and  implementation  mistakes  (bugs).  With  this  definition  of  errors,  uncertainties 
are  naturally  associated  to  the  choice  of  the  physical  models  and  to  the  specification  of  the  input  parameters 
required  for  performing  the  analysis.” 

To  properly  (or  faithfully)  characterize  the  uncertainty  in  each  space  object,  there  are  essentially  three 
interrelated  processes  that  one  must  consider  [3,  57,  69]. 

•  Verification  is  the  process  of  determining  how  accurately  the  computation  solves  the  underlying  equa¬ 
tions  of  the  model  and  other  quantities  of  interest. 

•  Validation  is  the  process  of  determining  the  degree  to  which  a  model  is  an  accurate  representation  of 
the  real  world  for  the  intended  uses  of  the  model. 


6 


2.1.  SOME  DEFINITIONS 


CHAPTER  2.  UNCERTAINTY  QUANTIFICATION 


•  Uncertainty  Quantification  is  the  process  of  determining  the  various  sources  of  errors  and  uncertain¬ 
ties,  properly  characterizing  these  errors  and  uncertainties,  and  the  roll-up  of  these  in  the  prediction 
of  the  quantities  of  interest. 

Sensitivity  analysis  (SA)  [57],  on  the  other  hand,  investigates  the  connection  between  inputs  and  outputs 
of  a  (computational)  model;  more  specifically,  it  allows  one  to  identify  how  the  variability  in  an  output 
quantity  of  interest  is  connected  to  an  input  in  the  model,  and  which  input  sources  will  dominate  the  response 
of  the  system.  On  the  other  hand,  uncertainty  analysis  aims  at  identifying  the  overall  output  uncertainty 
in  a  given  system.  The  main  difference  is  that  sensitivity  analysis  does  not  require  input  data  uncertainty 
characterization  from  a  real  device;  it  can  be  conducted  purely  based  on  the  mathematical  form  of  the  model. 
Large  output  sensitivities  (identified  using  SA)  do  not  necessarily  translate  into  important  uncertainties 
because  the  input  uncertainty  might  be  very  small  in  a  device  of  interest.  SA  is  often  based  on  the  concept 
of  sensitivity  derivatives,  the  gradient  of  the  output  of  interest  with  respect  to  input  variables.  The  overall 
sensitivity  is  then  evaluated  using  a  Taylor-series  expansion,  which,  to  first  order,  would  be  equivalent  to  a 
linear  relationship  between  inputs  and  outputs. 

An  important  subfield  of  UQ  is  Quantification  of  Margins  and  Uncertainties  (QMU),  a  methodology  that 
focuses  on  quantifying  the  ratio  of  a  system's  margin  (i.e.,  the  difference  between  the  measured  or  simulated 
output  of  a  system  and  a  performance  threshold)  to  its  uncertainty  (i.e.,  the  variability  in  the  output.)  QMU 
can  thus  be  used  to  estimate  the  probability  that  the  true  output  of  the  system  falls  within  the  performance 
threshold,  and  can  thus  be  used  for  decision  making.  An  application  of  QMU  in  SSA  might  be  determining 
the  probability  that  a  detected  maneuver  is  actually  a  true  maneuver,  or  if  it  is  a  false  alarm. 

Returning  to  the  subject  of  uncertainty  quantification,  uncertainties  are  generally  classified  as  either 
aleatoric,  epistemic  or  a  mixture  of  both.  Aleatoric  uncertainty  is  the  natural  randomness  or  physical 
variability  present  in  the  system  or  its  environment  and  is  thus  statistical  in  nature.  For  discrete  variables, 
the  randomness  is  parameterized  by  the  probability  of  each  possible  value.  For  continuous  variables,  the 
randomness  is  parameterized  by  a  probability  density  function.  In  contrast,  epistemic  uncertainty  is  un¬ 
certainty  that  is  due  to  limited  data  or  knowledge.  Epistemic  uncertainty  is  sometimes  referred  to  as  state 
of  knowledge  uncertainty,  subjective  uncertainty,  Type  B,  or  reducible  uncertainty,  meaning  that  the  uncer¬ 
tainty  can  be  reduced  through  increased  understanding  (research),  or  increased  and  more  relevant  data. 

Aleatoric  uncertainties  can,  by  definition,  be  represented  using  a  statistical  approach,  such  as  a  covari¬ 
ance  matrix  or  probability  density  function.  In  contrast,  epistemic  uncertainties  cannot  normally  be  directly 
represented  by  a  probability  density  function.  Non-probabilistic  approaches  include  evidence  (Dempster- 
Shafer)  theory,  possibility  theory,  fuzzy  set  theory,  and  interval  analysis.  However,  since  traditional  filtering 
methods,  such  as  Kalman  filtering,  require  the  states  and  parameters  to  be  represented  as  a  probability  dis¬ 
tribution,  it  is  often  necessary  to  approximate  the  epistemic  uncertainty  by  adding  aleatoric  uncertainty  to 
the  model. 

The  term  covariance  consistency  [76]  is  used  in  other  tracking  domains  in  place  of  the  term  covari¬ 
ance  realism,  which  is  the  proper  characterization  of  the  covariance  (statistical  uncertainty)  in  the  state  of 
a  system.  Covariance  realism  requires  that  the  estimate  of  the  mean  be  the  true  mean  (i.e.,  the  estimate  is 
unbiased)  and  the  covariance  possesses  the  right  size,  shape,  and  orientation  (i.e.,  consistency).  Relaxing 
Gaussian  assumptions,  uncertainty  realism  is  the  proper  characterization  of  the  uncertainty  in  that  state 
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using  a  general  (i.e.,  non-Gaussian)  probability  density  function.  Uncertainty  realism  requires  that  all  cu- 
mulants  (beyond  a  state  and  covariance)  be  properly  characterized.  The  relationship  between  covariance 
realism  and  uncertainty  realism  is  that  the  former  is  a  necessary  but  not  a  sufficient  condition  for  achieving 
the  latter.  The  two  definitions  coincide  if  the  process  is  Gaussian. 

In  addition  to  the  two  basic  types  of  uncertainties,  the  two  fundamental  problems  considered  in  uncer¬ 
tainty  quantification  are  the  forward  propagation  of  uncertainty  and  the  inverse  assessment  of  model  (e.g., 
dynamics,  space  environment,  and  measurement)  uncertainty  and  parameter  uncertainty. 

•  The  problem  of  forward  propagation  of  uncertainty  is  to  determine  the  uncertainties  in  the  state, 
model,  and  system  parameters  at  a  final  time  (e.g.  after  orbital  propagation).  These  uncertainties  are 
usually  represented  as  probability  density  functions.  For  astrodynamics,  this  uncertainty  is  surveyed 
in  Chapter  7. 

•  The  problem  of  inverse  uncertainty  quantification  is,  given  some  experimental  measurements  of  a  sys¬ 
tem  and  some  computer  simulation  results  from  its  mathematical  model,  to  estimate  the  discrepancy 
between  the  true  dynamics  of  the  system  and  the  mathematical  model  (which  is  called  bias  correc¬ 
tion),  and  to  estimate  the  values  of  unknown  parameters  in  the  model  if  there  are  any  (which  is  called 
parameter  calibration  or  simply  calibration).  Generally,  it  is  a  much  more  difficult  problem  than  un¬ 
certainty  propagation;  however,  it  is  of  great  importance  since  it  is  typically  implemented  in  a  model 
updating  process. 

2.2  Recommendations 

The  general  subject  of  “uncertainty  quantification”  is  an  important  area  of  research  and  development  within 
the  engineering  and  science  community.  Many  of  the  objectives  in  achieving  the  quantification  of  errors 
and  uncertainties  in  this  field  are  directly  applicable  to  those  in  astrodynamics.  While  most  of  the  literature 
on  uncertainties  in  astrodynamics  is  focused  on  aleatoric  uncertainty,  methods  and  algorithms  for  epistemic 
uncertainty  quantification  should  be  investigated. 
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Survey  of  Some  of  the  Mission  Areas 


The  improvement  or  achievement  of  realistic  covariances  and  more  generally  a  realistic  uncertainty  realism 
in  the  characterization  of  the  errors  in  the  state  of  a  space  object  is  central  to  at  least  four  areas: 

•  computation  of  the  probability  of  collision  for  conjunction  assessment, 

•  data  or  track  association/correlation, 

•  maneuver  detection, 

•  sensor  tasking  and  scheduling. 

In  the  following,  these  four  areas  arc  briefly  discussed.  The  primary  goal  is  to  explain  the  impact  of  having 
an  unrealistic  covariance  (or,  more  generally,  an  unrealistic  PDF)  that  characterizes  the  state  of  an  object. 

3.1  Conjunction  Assessment 

3.1.1  Role  of  Covariance  in  the  Calculation  of  Probability  of  Collision  Parameter 

There  arc  two  methods  typically  employed  to  calculate  the  probability  of  collision  (Pc),  which  is  the  main 
conjunction  analysis  and  risk  assessment  parameter  between  two  space  objects.  The  first  is  used  when  the 
duration  of  the  conjunction  is  extremely  short,  while  the  second  is  used  otherwise. 

3. 1.1.1  Short-Duration  Method 

When  the  duration  of  the  conjunction  is  extremely  short,  rectilineal-  motion  of  the  two  spacecraft  and  invari¬ 
ant  covariances  can  be  assumed;  and  it  is  thus  possible  to  reduce  the  problem  from  a  3-  to  2-dimensional  (or 
lower)  formulation.  This  is  performed  as  follows: 

•  The  system  of  both  objects  is  propagated  to  the  time  of  closest  approach  (TCA)  and  projected  into 
a  plane  orthogonal  to  the  relative  velocity  vector  (Vps  and  y-axis  in  this  diagram)  between  the  two 
objects-this  is  called  the  conjunction  plane, 

•  The  covariances  for  the  primary  object  (the  protected  asset)  and  the  secondary  object  (the  conjunctor) 
are  added  together  to  form  a  combined  covariance, 
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•  A  coordinate  system  is  created  with  the  position  of  the  secondary  object  at  the  origin,  the  x  axis 
positioned  along  the  relative  position  vector  at  TCA  (i.e.,  the  primary  object  is  located  on  the  positive 
x-axis),  and  the  z  axis  in  the  conjunction  plane  and  orthogonal  to  the  x  axis, 

•  A  Gaussian  PDF  is  created  whose  center  is  the  origin  and  whose  covariance  is  the  combined  covari¬ 
ance,  and 

•  A  “region  of  conjunction”  A  is  created  whose  center  is  the  position  of  the  primary  object  and  whose 
radius  is  large  enough  to  circumscribe  both  the  primary  and  secondary  objects’  sizes  (this  radius  is 
referred  to  as  the  “hard-body  radius,”  or  FIBR). 

A  diagram  of  this  encounter  region  (taken  from  [49])  is  given  in  Figure  3.1.1: 

Figure  3.1.1  Conjunction  duration  as  represented  in  the  conjunction  plane 


z 


The  probability  of  collision  is  then  the  integral  of  the  PDF  over  the  region  of  conjunction,  which  can  be 
calculated  as  follows  (this  calculation  may  require  a  rotation  of  axes  in  the  conjunction  plane  to  remove 
undesirable  correlation  terms;  this  can  be  done  without  loss  of  generality): 

Pc  =  — .  ^  =  [[  exp  (  —  -rTC*~1r  )  dXdZ  (3.1.1) 

V(2tt)2|C*|  JJa  \  2  J 

in  which  r  =  [x,  z]  and  C*  is  the  two-dimensional  covariance  matrix  that  results  from  projecting  the 
three-dimensional  position  covariance  matrix  C  into  the  conjunction  plane.  This  is  the  form  of  simplified 
evaluation  promoted  by  Foster  [90,  91];  other  methods  that  are  similar  include  those  by  Patera  [219],  Chan 
[49],  and  Alfano  [4,  5].  While  these  approaches  may  differ  in  the  dimensionality  of  the  solution,  all  of 
them  hold  a  similarly  prominent  role  for  the  combined  covariance  and  its  assumed  Gaussian  behavior.  We 
remark  that  the  rectilinear  assumption  used  in  these  short-duration  methods  can  be  relaxed  as  described,  for 
example,  by  Vittaldev  and  Russell  [273]. 
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3. 1.1.2  Long-Duration  Method 

For  those  situations  in  which  the  short-duration  assumption  does  not  inhere,  some  simplified  three-dimensional 
methods  (e.g.,  [200])  or  Gaussian  mixture  models  (e.g.,  [65,  273])  have  been  proposed;  but  typical  opera¬ 
tional  procedure  is  to  make  use  of  Monte  Carlo  techniques.  These  approaches  can  be  conducted  in  two 
ways,  the  less  computationally  burdensome  of  which  is  as  follows: 

•  Propagate  both  the  primary  and  secondary  states  and  covariances  to  TCA, 

•  Draw  Monte  Carlo  samples  from  each  of  these  distributions, 

•  Calculate  a  new  TCA  and  miss  distance*  for  each  pair  of  draws,  and 

•  Determine  whether  this  miss  distance  falls  within  the  hard-body  radius. 

The  execution  of  a  large  number  of  such  trials  will  determine  a  synthetic  Pc  (number  of  trials  that  fall 
within  F1BR  /  total  number  of  trials).  This  option  will  address  the  failure  of  rectilinear  dynamics  to  represent 
the  actual  conjunction  situation,  but  it  still  relies  on  a  realistic  covariance  in  the  region  near  TCA.  The  more 
computationally-intensive  but  higher-accuracy  approach  is,  instead: 

•  Draw  Monte  Carlo  samples  from  the  primary  and  secondary  covariances  at  their  epoch  times. 

•  Propagate  each  of  these  samples  forward  to  a  new  TCA  and  calculate  the  miss  distance,  and 

•  Determine  whether  the  miss  distance  falls  within  the  hard-body  radius. 

This  approach  remediates  both  shortcomings  of  rectilinear  dynamics  in  the  region  of  the  conjunction 
and  non-linearities  in  the  propagation  of  the  individual  objects’  covariances  to  TCA. 

3.1.2  Effects  of  Covariance  Ir realism 
3.1.2.1  Covariance  Size 

Figure  3.1.1  shows  that  the  density  of  the  Gaussian  PDF  in  its  overlap  of  the  region  of  conjunction  will 
determine  the  value  of  Pc ■  Since  the  PDF  has  support  over  the  entire  conjunction  plane,  there  is  always 
overlap  with  the  region  of  conjunction;  the  question  is  whether  the  overlap  occurs  in  a  relatively  dense  or 
sparse  region  of  the  PDF.  One  can  see  that  the  relative  size  of  the  covariance  and  HBR  region,  and  their 
relative  placement,  will  affect  the  Pc  calculation.  For  a  fixed  miss  distance  (which  is  the  distance  from  the 
center  of  the  PDF  to  the  center  of  the  region  of  conjunction  along  the  x-axis),  we  see  that: 

•  If  the  covariance  is  very  tight  (principal  axis  much  smaller  than  the  miss  distance),  the  Pc  is  low 
because  the  conjunction  region  lies  in  the  tails  of  the  PDF; 

•  If  the  covariance  is  medium  size  (principal  axis  approximately  equal  to  the  miss  distance),  the  Pc  is 
larger;  and, 

•  If  the  covariance  is  very  loose  (principal  axis  much  larger  than  the  miss  distance)  the  Pc  becomes 
smaller  again  because  the  PDF  becomes  spread  out  over  a  larger  region,  and  thus  has  lower  density. 

*  In  point  of  fact,  the  miss  distance  and  covariance  are  themselves  linked;  so  the  imposition  of  a  “fixed  miss  distance”  while 
the  covariance  is  altered  is  not  a  realistic  decision-support  activity  in  an  operational  environment.  However,  it  is  helpful  as  an 
illustrative  sensitivity  analysis. 
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Figure  3.1.2,  which  plots  Pc  as  a  function  of  the  ratio  of  the  combined  covariance’s  lcr  value  to  the  miss 
distance,  shows  this  effect.  For  simplicity,  a  spherical  covariance  (projected  into  the  conjunction  plane  as 
a  circle)  is  used  here,  with  a  series  of  contours  showing  behavior  at  hard  body  radii  from  1%  to  5%  of  the 
miss  distance.  One  can  observe  that  the  sensitivity  of  Pc  to  covariance  size  can  be  substantial  or  relatively 
minor  depending  on  where  on  the  curve  the  nominal  value  happens  to  fall:  in  some  places  the  slope  is 
steep;  in  others  rather  flat.  However,  it  is  clear  that  there  are  places  on  this  curve  where  covariance  over-  or 
under-reporting  by  a  factor  of  2  or  3  can  change  the  resulting  Pc  by  more  than  an  order  of  magnitude,  the 
commonly-used  rule-of-thumb  for  considering  a  change  in  Pc  to  be  significant. 

Figure  3.1.2  Pc  as  a  function  of  sigma  /  miss  distance  ratio,  with  HBR  contours 


3.1.2.2  Covariance  Orientation 

The  angular  orientation  of  the  covariance  in  the  conjunction  plane,  especially  as  the  covariance  moves  from 
circularity  to  a  more  oblong  ellipse,  can  have  a  considerable  effect  on  the  Pc .  Figure  3.1.3  shows  Pc  as 
a  function  of  the  “clock  angle”  in  the  conjunction  plane  (angular  deviation  of  the  principal  axis  from  the 
z-axis;  a  90°  clock  angle  has  the  highest  Pc  because  it  means  that  the  principal  axis  is  aligned  with  the 
x  axis)  for  a  variety  of  covariance  oblateness  values  (minor  axis  from  10%  of  principal  axis  to  100%  of 
principal  axis,  with  100%  indicating  a  circular  covariance;  miss  distance  of  unity  with  covariance  principal 
axis  also  of  unity  and  HBR  of  2%  of  miss  distance). 

As  one  can  see,  the  effect  of  covariance  orientation  on  Pc  can  be  significant,  and  the  degree  of  this 
effect  is  substantially  heightened  with  increasing  covariance  oblateness.  There  are  certainly  regions  in  which 
relatively  small  changes  in  orientation  can  affect  the  Pc  by  the  order-of-magnitude  threshold  of  significance. 
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Figure  3.1.3  Pc  vs  covariance  orientation  in  conjunction  plane.  Contours  show  oblateness  of  covariance. 


3. 1.2.3  Covariance  Shape 

A  number  of  researchers  have  demonstrated  that  even  initially  Gaussian  error  volumes,  with  their  familiar 
ellipsoidal  shape,  can  become  non-Gaussian  over  days  or  even  hours  of  propagation  [64,  122,  149,  158,  218, 
231].  This  effect  is  due  to  the  fact  that  orbit  positional  evolution  (and  thus  error  mapping)  is  curvilinear 
in  nature  and  thus  not  suited  to  a  Cartesian  framework.  The  error  manifests  itself  in  Cartesian  space  as  a 
curved  ellipsoid,  or  “bananoid,”  as  shown  by  the  blue  Monte  Carlo  samples  in  Figure  3.1.4,  as  opposed  to 
the  red  covariance  ellipse,  which  is  both  incorrect  in  shape  and  misaligned  with  the  nominal  velocity  vector 
(the  object  represented  here  is  a  near-circular  orbit  with  an  altitude  of  1400km). 

While  from  the  figure  the  situation  may  appear  severe,  one  must  consider  both  (1)  the  fact  that  in  Figure 
3.1.4,  the  Y  axis  scale  is  stretched  vertically  by  a  factor  of  25  relative  to  the  X  axis  scale  to  accentuate 
illustratively  the  shape  differential  between  the  two  error  regions,  and  (2)  that  the  two  regions  do  overlap 
in  the  area  of  greatest  density.  This  latter  phenomenon  was  identified  by  Ghrist  and  Plakalovic  [99],  who 
have  shown  that  this  non-Gaussian  curvature  appears  to  have  little  effect  on  the  Pc  for  high- Pc'  events  (i.e. 
Pc  >  10~4,  which  is  the  usual  threshold  for  active  remediation  with  conjunction  avoidance  maneuvers)  and 
that,  when  it  does  appeal-,  it  can  often  be  addressed  by  a  Monte  Carlo  evaluation  from  a  post-propagation 
transformation  to  equinoctial  element  space.  As  such,  while  it  is  always  desirable  to  employ  the  true  shape 
of  the  eiTor  region  for  the  Pc  calculation,  the  issue  of  a  non-ellipsoidal  covariance  shape  is  a  less  urgent  is¬ 
sue  for  conjunction  analysis  risk  assessment  than  proper  sizing  and  orientation  of  the  ellipsoidal  covariance. 
Nonetheless,  while  it  may  be  the  case  that  curvilinear  bending  of  the  true  covariance  will  often  not  influence 
the  resultant  Pc  calculation  appreciably,  the  wholesale  under-  or  over-sizing  and  mis-orientation  of  the  co- 
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Figure  3.1.4  “Bananoid”  covariance  as  compared  to  ellipse  from  propagated  covariance 


variance,  as  stated  previously,  can  have  a  significant  effect  on  the  Pc.  To  this  end,  it  is  quite  important  for 
the  conjunction  assessment  problem  to  make  all  reasonable  efforts  to  obtain  realistically  sized  and  oriented 
covariances. 

3.1.3  Conjunction  Assessment  Covariance  Realism  Efforts  to  Date 
3.1.3.1  JSpOC  Inherent  Features 

Since  the  SP  catalog  maintenance  software  presently  in  use  at  the  JSpOC  was  developed  for  CA  purposes, 
it  includes  features  that  were  introduced  to  improve  covariance  realism  for  this  particular  application.  The 
first  of  these  is  the  implementation  of  a  consider  parameter  to  improve  covariance  realism  in  propagation. 
Release  14-3  of  the  SP  satellite  catalogue  software  implemented  functionality  to  calculate  a  dynamic  con¬ 
sider  parameter  that  would  attempt  to  compensate  for  atmospheric  density  mismodeling  and  a  non-static 
satellite  frontal  area.  This  calculated  parameter,  the  specifics  of  which  are  described  in  more  detail  below,  is 
added  to  the  ballistic  coefficient  variance  in  the  unpropagated  covariance  matrix;  and  through  the  pre-  and 
post-multiplication  of  the  state-transition  matrix  and  the  ballistic  coefficient  correlation  terms,  this  increase 
affects  the  other  covariance  elements. 

The  Dynamic  Consider  Parameter  comprises  two  components.  One  component  is  due  to  the  global 
eiTor  in  the  atmospheric  density  forecast.  This  component  will  vary  for  all  satellites  with  changes  in  the 
atmospheric  density  forecast.  The  other  component  is  due  to  satellite-specific  frontal  area  variation  in  the 
prediction  of  the  ballistic  coefficient.  This  component  will  change  as  the  frontal  area  changes  over  time  due 
to  rotation  of  the  satellite.  The  DCP  variance  is  the  sum  of  the  variance  of  these  two  components. 

The  percent  RMS  error  in  the  atmospheric  density  forecast  is  parameterized  as  a  function  of  height  above 
the  surface  of  the  Earth  and  geomagnetic  activity.  The  main  source  of  atmospheric  density  prediction  errors 
is  due  to  the  sudden  short  term  (less  than  3  days)  geomagnetic  storms,  characterized  by  the  geomagnetic 
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index  ap  and  the  disturbance  storm  time  index  Dst.  The  Jacchia-Bowman-HASDM  2009  (JBH09)  model 
uses  the  ap  index  for  low  storm  activity  and  the  Dst  index  during  moderate  to  high  storm  activity.  The  percent 
RMS  error  in  atmospheric  density  forecast  is  obtained  by  comparing  the  predictions  from  the  JBH09  model 
with  the  true  densities  computed  using  real  temperature  corrections  from  the  High  Accuracy  Satellite  Drag 
Model  (HASDM).  The  frontal  area  variation  is  obtained  by  quantifying  the  ballistic  coefficient  error  via  their 
histories  by  satellite,  going  back  in  time  up  to  a  year.  This  process  includes  a  preprocessor  to  handle  outliers 
and  regularize  the  ballistic  coefficient  histories  to  even-day  values  for  subsequent  reduction  of  persisted 
ballistic  coefficient  error. 

The  second  feature  to  improve  covariance  realism  in  the  SP  catalog  maintenance  software  is  the  scaling 
of  the  entire  covariance  by  the  weighted  RMS  of  the  batch  differential  correction  at  propagation-time.  The 
covariance  that  appears  in  the  vector  covariance  message  (VCM)  is  not  scaled  this  way;  but  when  propa¬ 
gation  is  conducted  by  the  operational  system  (and  the  SP  astrodynamics  standard),  the  entire  covariance 
is  multiplied  by  the  square  of  the  weighted  RMS  if  it  is  greater  than  unity.  There  are  apparently  some 
theoretical  arguments  for  why  this  is  a  statistically-sanctioned  procedure,  but  there  is  no  known  published 
explanation  that  can  be  cited  here. 

3.1.3.2  Scale-Factor  Computations  for  JSC  ISS  Conjunction  Assessment 

It  was  recognized  early  in  the  history  of  the  ISS  protection  mission  that  secondary  object  covariances  were 
in  general  undersized  and  that  some  sort  of  covariance  scaling  approach  would  be  required.  JSC  performed 
analyses  to  compute  a  single  scale  factor  value  for  each  propagation  point  of  interest;  before  using  the 
secondary  object’s  covariance  in  conjunction  analysis  activities,  the  entire  matrix  would  be  multiplied  by  this 
scalar  value.  The  scale  factor  determination  analysis  compared,  for  a  select  number  of  satellites,  propagated 
states  to  reference  ephemerides  to  determine  position  difference  statistics.  If  these  errors  are  Gaussian  in 
distribution  and  properly  represented  by  the  propagated  covariances,  then  the  entire  family  of  such  position 
differences  should  obey  the  following  relationship: 

rTC~lr  =  xldof  (3.1.2) 

in  which  r  is  the  vector  of  position  differences  between  the  reference  and  propagated  orbit  and  C  is  the 
position  portion  of  the  combined  primary  and  secondary  covariances,  propagated  to  the  time  of  closest 
approach.  The  left  side  of  Equation  (3.1.2)  represents  an  entire  set  of  points  for  which  this  computation  has 
been  made,  and  the  right  side  indicates  that  it  should  conform  to  a  chi-squared  distribution  with  three  degrees 
of  freedom.  The  scale  factor  to  be  used,  therefore,  is  one  that  brings  the  sample  distribution  as  close  to  a 
3-dof  chi-squared  distribution  as  possible.  The  approach  used  by  JSC  was  to  perform  visual  comparisons 
of  CDF  plots  and  make  the  determination  this  way.  While  some  outlier  rejection  was  performed,  this  is 
still  often  a  superior  approach  in  the  presence  of  outliers,  which  tend  to  derail  formal  goodness-of-fit  tests. 
Subsequent  to  their  original  efforts,  JSC  has  also  established  a  procedure  to  determine  axis-by-axis  scale 
factors  (with  each  set  proper  to  different  propagation  points).  Since  their  original  operational  tool  could 
only  accept  a  single  scale  factor,  their  current  operational  practice  is  to  run  this  tool  (with  a  single  factor) 
and  then  compare  the  results  to  an  off-line  utility  that  uses  the  axis-by-axis  scale  factors;  and  anecdotal 
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evidence  does  not  document  a  significant  deviation  between  the  two  approaches  (i.e.,  the  resultant  values 
of  Pc  differ  by  less  than  an  order  of  magnitude).  The  original  calculation  methodology  and  results  are 
documented  in  [92],  and  an  updated  methodology  and  set  of  scale  factors  produced  after  the  operational 
installation  of  the  HASDM  functionality  is  given  in  [153]  and  [154]. 

3.1.3.3  Aerospace  Corporation  Full-Correction  Matrices 

The  Aerospace  Corporation  has  for  several  years  been  pursuing  an  approach  to  provide  full  corrections  of 
the  position  portion  of  covariance  matrices.  Their  approach  is  to  develop  correction  matrices  that,  when  pre- 
and  post-multiplied  by  an  eigenvalue/eigenvector  decomposed  version  of  the  propagated  covariance  matrix, 
will  correct  all  six  elements  of  that  matrix  to  reflect  proper  scaling  in  all  three  axes  and  proper  angular 
orientation  of  the  covariance  axes.  State  vectors  and  covariances  (from  VCMs)  are  propagated  to  the  epoch 
times  of  future  VCMs  and  position  comparisons  performed;  and  an  elaborate  technique,  documented  in 
[47]  and  [48],  decomposes  the  set  of  matrices  to  be  corrected  into  their  individual  components  and  forms  a 
large  linear  problem  with  which  to  solve  for  the  optimal  value  for  each  component  of  the  inner  correction 
matrix.  A  useful  adjunct  to  this  solution  is  the  ability  to  determine  confidence  bounds  on  each  correction 
element;  these  confidence  bounds  can  be  used  to  produce,  say,  a  5th  percentile  corrected  covariance  and 
a  95th  percentile  corrected  covariance;  these  two  corrected  covariances,  along  with  the  nominal  corrected 
covariances,  can  then  be  used  in  three  separate  Pc  calculations  to  give  a  5th,  50th,  and  95th  percentile  Pc 
value.  A  confidence-region-enabled  Pq  calculation  would  be  a  very  useful  addition  to  conjunction-related 
risk  analysis. 

The  Aerospace  approach  has  sustained  a  validation  activity,  with  NASA/GSFC  serving  as  the  indepen¬ 
dent  validation  agency;  the  validation  report  [111]  has  been  completed,  but  unfortunately  is  available  only 
to  a  restricted  distribution.  These  correction  matrices,  which  are  calculated  for  each  individual  satellite  and 
classes  of  satellites,  all  at  a  series  of  propagation  points,  were  made  available  to  certain  customer  sets  in  the 
fall  of  2014. 

3.2  Data  Association/Correlation  for  UCT  Resolution  and  Catalog  Mainte¬ 
nance 

The  data  association  or  correlation  problem  is  that  of  determining  which  sensor  report  goes  with  which 
space  object.  The  fusion  problem  is  that  of  combining  information  from  one  or  more  sensors  to  improve  the 
state  of  an  object.  These  two  aspects  are  inseparable  parts  of  the  same  problem  and  both  impact  covariance 
or  uncertainty  realism.  (Cross-tags  degenerate  the  covariance  as  discussed  in  Section  1.1.  The  probability  of 
association  is  used  to  better  quantify  the  likelihood  of  cross-tags  or  misassociations.)  Likewise,  an  unrealistic 
covariance  can  lead  to  cross-tags  (i.e.,  misassociations).  Before  illustrating  the  sensitivity  to  unrealistic 
covariances,  a  brief  review  of  data  association  methods  is  presented. 

Data  association  or  correlation  methods  for  multiple  target/object  tracking  divide  into  two  broad  classes, 
namely,  single  frame  methods  and  multiple  frame  methods.  The  single  frame  methods  include  nearest  neigh¬ 
bor,  covariance  based  track  association  (CBTA)  [6,  1 14,  1 16],  single  hypothesis  or  global  nearest  neighbor 
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based  on  a  two-dimensional  assignment  problem  formulation  [38],  and  joint  probabilistic  data  association 
(JPDA)  (see,  e.g.  [25]  and  the  extensive  references  therein).  Nearest  neighbor  works  well  for  widely-spaced 
objects,  but  can  degenerate  significantly  as  object  spacing  decreases.  Global  nearest  neighbor  extends  the 
validity  of  nearest  neighbor  into  the  regime  of  closely  spaced  objects  and  works  well  in  many  track  mainte¬ 
nance  and  low  clutter  environments.  JPDA  was  developed  as  an  algorithm  for  track  maintenance  of  a  few 
targets  in  dense  clutter  and  has  been  highly  successful  for  such  problems.  For  closely  spaced  objects  in 
dense  clutter  environments,  JPDA  suffers  from  the  coalescence  problem  [37,  38].  JPDA  mitigates  misasso- 
ciations,  i.e.,  cross-tags,  by  averaging  all  the  measurements  within  a  neighborhood  of  the  projected  track. 
A  recent  article  by  Stauch,  Baldwin,  Jah,  Kelecy  and  Hill  [248]  demonstrates  JPDA  for  space  surveillance. 
The  most  successful  of  the  multiple  frame  methods  are  multiple  hypothesis  tracking  (MHT)  and  multi¬ 
ple  frame  assignment  (MFA),  which  is  an  optimization  based  MHT  [224].  MHT/MFA  methods  mitigate 
misassociation  (cross-tags)  by  providing  an  opportunity  to  change  past  decisions  to  improve  current  ones 
or,  equivalently,  holding  ambiguous  association  decisions  in  abeyance  until  more  information  is  available. 
MHT/MFA  methods  work  especially  well  at  the  system  level  and  in  low  to  moderate  clutter  environments, 
but  not  necessarily  in  the  domain  of  heavy  clutter  found  at  the  sensor  level.  In  dense  tracking  environments 
such  as  LEO  breakups  or  GEO  clusters  with  closely-spaced  objects,  the  performance  improvements  of  mul¬ 
tiple  frame  methods  over  single  frame  methods  are  very  significant.  (The  papers  by  Aristoff,  Horwood, 
Singh,  Poore,  Sheaff,  and  Jah  [17,  244,  244]  demonstrates  MHT  for  space  surveillance.)  In  dense  clutter 
environments  where  cross-tags  are  present,  JPDA  updates  to  the  state  covariance  are  found  to  improve  co- 
variance  realism  for  single  and  multiple  hypothesis  tracking  [74].  For  heavy  clutter  environments,  JPDA 
methods  and  those  based  on  finite  set  statistics  (FISST)  [187]  may  be  more  appropriate. 

Each  of  the  above  methods  makes  use  of  a  statistical  distance  as  an  association  metric  whether  it  be 
track-to-track  or  sensor  measurement  to  track,  which  is  in  turn  based  on  a  probability  density  function.  For 
example,  CBTA  [6,  1 14,  116]  uses  the  Mahalanobis  distance  (squared).  For  Gaussian  processes,  single  and 
multiple  hypothesis  tracking  methods  use  a  negative  log  of  a  likelihood  ratio,  a  component  of  which  is  a 
Mahalanobis  metric  (or  distance  squared)  [224].  To  demonstrate  the  importance  of  the  covariance  matrix 
in  data  association,  consider  the  case  of  CBTA  in  which  one  measures  the  statistical  distance  between  two 
objects  using  the  Mahalanobis  metric  (see  Alfriend  [6],  [1 16]) 

M2  =  (®!  -  x2f  (Pi  +  P2)— 1  (®1  -  x2) . 

In  this  formulation,  we  have  M2  <  %2  (6). 

To  demonstrate  how  sensitive  the  Mahalanobis  distance  (squared)  can  be  to  the  right  size  and  shape  of 
the  covariance,  suppose  M2  =  %205(6),  then  5 M2  =  Xo  0002(6)-  Thus  decreasing  the  size  of  the  covariance 
by  a  factor  of  5  leads  to  a  change  of  3  orders  of  magnitude  in  the  confidence  interval.  In  general,  one  would 
expect  a  value  of  22.5  for  a  99.9%  confidence  interval.  However,  in  one  study  [116],  22%  of  the  objects 
had  a  Mahalonobis  distance  squared  (M2)  value  that  were  greater  than  100.  This  problem  can  be  mitigated 
by  inflating  the  covariances;  however,  if  the  covariance  is  too  large,  then  for  closely  spaced  objects  the  as¬ 
sociations  become  ambiguous  and  cross-tags  result.  Thus,  having  a  realistic  covariance  and  more  general 
uncertainty  is  essential  for  good  association.  Although  this  discussion  applies  to  sensor  track-to-track  asso¬ 
ciation  in  the  presence  of  no  auto  or  cross  correlation,  the  same  is  true  of  associating  radar  measurements  or 
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EO  measurements  to  system  tracks  or  orbits.  Indeed,  having  realistic  covariances  is  perhaps  the  single  most 
important  need  if  one  is  to  achieve  a  superior  space  surveillance  capability. 

3.3  Maneuver  Detection 

Maneuver  detection  in  cataloged  space  objects  and  identification  of  modeling  errors  for  these  objects  are 
two  sides  of  the  same  coin.  While  each  of  these  usually  have  distinctive  signatures,  there  is  much  room  for 
ambiguity  between  the  two.  Thus  we  will  treat  them  similarly  in  our  discussion.  The  question  at  hand  is 
what  the  impact  of  covariance  realism  for  maneuver  detection  or  model  error  identification. 

Independent  of  mapping  non-linearities  or  even  the  non-Gaussian  nature  of  an  object’s  probability  dis¬ 
tribution  function,  a  key  component  of  orbit  determination  and  prediction  is  the  2nd  moment  of  an  object’s 
uncertainty  distribution,  or  the  covariance.  While  every  object  has  a  covariance,  it  is  technically  unknow¬ 
able  at  any  instant  and  must  be  predicted  from  models  or  inferred  from  an  aggregate  of  observations  of  a 
similar  event.  This  issue  is  at  the  heart  of  covariance  realism,  and  is  what  makes  this  problem  fundamentally 
difficult.  The  most  commonly  used  covariance  for  an  object  is  the  “formal  covariance”  that  arises  from  a 
least-squares  orbit  determination  process,  in  its  most  simple  form  just  based  on  the  assumed  covariance  val¬ 
ues  of  the  measurements.  Such  formal  covariances  arc  generally  optimistic,  and  for  enough  measurements 
will  generally  over  estimate  the  degree  of  certainty  in  an  object’s  state.  This  holds  true  even  if  non-linearities 
are  accounted  for  in  mapping  the  uncertainty  distribution,  such  as  via  an  unscented  transformation  [156]  or 
higher-order  state  transition  tensors  [218]. 

An  over  estimate  of  certainty,  or  a  too-small  covariance,  is  a  core  problem  when  wishing  to  detect 
maneuvers  or  model  errors.  If  the  true  uncertainty  of  the  object  is  larger  than  the  predicted  covariance, 
then  algorithms  that  use  the  predicted  covariance  to  identify  statistically  rare  events  as  an  indication  of  a 
maneuver  will  generate  many  false  alarms,  and  thus  will  become  untrustworthy.  In  essence,  the  maneuver 
detection  algorithm  will  be  detecting  statistical  variations  as  an  indication  of  an  actual  event. 

There  are  two  main  ways  to  deal  with  this  problem,  and  in  many  cases  both  can  be  applied  to  obtain 
improved  computed  covariances.  One  approach  is  to  better  model  the  uncertainties  associated  with  the 
known  physical  forces,  measurement  biases,  and  other  effects  that  can  influence  the  dynamics  of  an  object 
or  the  measurements  of  that  object.  In  this  approach,  sometimes  called  the  “Consider  Covariance,”  the 
effects  of  known  uncertainties  in  other  parameters  that  affect  the  trajectory  and  measurements  of  the  space 
object  are  formally  added  to  the  formal  covariance  which  comes  out  of  an  orbit  determination  filter  (c.f., 
[254]).  For  Gaussian  processes,  the  covariance  in  the  object’s  state  due  to  these  uncertain  parameters  is  just 
added  to  the  formal  covariance  of  the  state  due  to  measurements  and  mapping  alone.  Carrying  out  such  an 
analysis  requires  exacting  models  of  the  dynamics  and  measurement  systems,  yet  can  produce  very  accurate 
corrections  to  the  covariance  when  the  physics  of  these  effects  and  the  associated  uncertainties  in  their 
parameters  are  well  understood.  A  proper  covariance  analysis  can  even  be  used  to  determine  when  certain 
parameters  that  would  not  normally  be  estimated  have  sufficient  information  content  in  the  measurements 
to  be  better  estimated.  This  approach  to  producing  improved  covariance  estimates  is  widely  used  in  the 
interplanetary  spacecraft  navigation  community  and  has  been  highly  refined.  When  properly  defined,  a 
consider  covariance-motivated  approach  can  be  an  effective  tool  in  developing  covariances  that  can  be  used 
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for  maneuver  detection  or  model  improvement. 

Despite  the  development  of  accurate  physical  models,  the  consider  covariance  approach  generally  has 
its  limits,  and  ultimately  our  ability  to  model  and  know  all  the  relevant  physical  forces  acting  on  an  object 
have  a  limit.  For  interplanetary  spacecraft,  this  limit  is  often  only  reached  for  accelerations  on  the  order  of 
1  x  10“ 13  km/s2.  For  Earth  orbiting  satellites,  however,  our  ability  to  gain  this  level  of  modeling  precision  is 
generally  limited  due  to  the  atmospheric  drag  and  solar  radiation  pressure  effects.  This  limitation  motivates 
the  other  approach  that  is  often  used  to  improve  covariance  computation. 

If  the  physics  of  the  environment  are  not  well  modeled  or  understood,  or  if  the  occurrence  of  a  ran¬ 
dom  event  acting  on  the  state  is  considered  to  be  high,  the  covariance  can  be  presumed  to  be  acted  on  by 
stochastic  accelerations  as  a  function  of  time.  Incorporating  this  into  the  uncertainty  model  adds  to  the 
formal  covariance  and  causes  the  overall  uncertainty  to  be  higher.  This  is  the  approach  that  has  been  more 
commonly  applied  in  the  literature  and  described  below. 

3.3.1  Previous  Work 

Typical  methods  for  dealing  with  uncertain  dynamics  in  the  state  estimation  process  include  adding  pro¬ 
cess  noise  to  the  system.  Dynamic  Model  Compensation  (DMC),  or  appending  dynamics  parameters  to  the 
state  vector  (citeTaScBo2004,Jazwinl998.  Process  noise,  while  effective  at  preventing  divergence  due  to 
mismodeling,  only  masks  the  problem.  It  provides  no  method  for  estimating  the  mismodeling,  detecting  its 
presence,  it  does  not  have  a  strong  physical  significance,  and  the  uncertainty  it  injects  in  the  system  limits 
the  accuracy  of  the  state  estimates.  DMC  requires  a  significant  amount  of  tuning,  and  the  type  of  dynamics  it 
can  replicate  are  limited  by  the  Gauss-Markov  functional  form  it  assumes.  Appending  dynamics  parameters 
to  the  state  can  be  an  effective  method  for  recovering  dynamics,  but  it  requires  a  known  model  for  those 
mismodeled  dynamics. 

Beyond  these  classical  methods,  the  problems  of  dynamic  mismodeling  identification  (maneuver  de¬ 
tection)  and  mismodeling  estimation  (maneuver  characterization/reconstruction  and  natural  dynamics  esti¬ 
mation)  have  been  addressed  for  different  systems.  Generally,  algorithms  have  been  developed  for  highly 
dynamical  systems  (e.g.  missile  tracking  and  guidance)  that  are  data-rich  (i.e.  observations  taken  through¬ 
out  a  maneuver).  Methods  such  as  Bar-Shalom  and  BirmiwaTs  Variable  Dimension  Filter  [24]  and  Chan, 
Hu,  and  Plant’s  Input  Estimation  Method  [50]  directly  append  accelerations  to  the  state  vector  for  estima¬ 
tion  when  a  maneuver  is  detected  through  residuals,  but  such  methods  require  observation  throughout  a 
continuous  maneuver.  Patera’s  space  event  detection  method  [220]  focuses  more  on  quick  events  in  an  as- 
trodynamics  context,  so  it  tends  to  neglect  smaller  maneuvers  and  natural  dynamics  mismodeling  as  well  as 
being  limited  in  application.  Lemmens  and  Krag  [175]  addressed  maneuver  detection  for  LEO  orbits  with  a 
Two-Line-Element-based  method.  Aaron  [2]  and  Folcik,  Cefola,  and  Abbot  [89]  addressed  maneuver  detec¬ 
tion  for  Geosynchronous  (GEO)  orbits  with  a  method  based  on  application  of  the  Extended  Semi-analytic 
Kalman  Filter  (ESKF).  These  three  methods  produce  promising  maneuver  detection  results  for  astrodynam- 
ics  applications,  but  they  provide  no  information  about  the  underlying  maneuvers.  Aside  from  DMC  and 
appending  parameter  to  the  state  vector  in  Kalman  filtering,  a  method  for  estimating  dynamics  parameters 
is  provided  by  Mandankan,  Singla,  Singh,  and  Scott  [190],  This  method  relies  on  polynomial  chaos.  It 
provides  both  state  and  parameter  estimates,  but  just  like  other  methods  it  requires  a  model  for  mismodeled 
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perturbations  and  it  does  not  have  maneuver  detection  properties.  Each  of  the  methods  described  does  not 
fully  address  the  problem  of  detecting  and  estimating  mismodeled  dynamics  in  data-sparse  systems.  We 
seek  an  algorithm  that  will  fully  address  the  entirety  of  this  problem  in  as  automatic  a  fashion  as  possible. 
To  accomplish  this,  we  first  must  determine  a  framework  that  will  allow  us  to  approach  each  aspect  of  the 
problem. 

The  problems  of  object  correlation,  maneuver  detection,  and  maneuver  characterization  in  data  sparse 
environments  were  addressed  by  Holzinger,  Scheeres,  and  Alfriend  [120]  with  a  distance  metric  approach 
that  is  based  on  optimal  control  policies.  They  develop  a  concept  called  control  distance  metrics  that  measure 
the  integrated  control  effort  that  connects  two  states  at  different  times  with  a  given  dynamical  model.  The 
distance  metric  they  introduced  is  an  unweighted  integral  of  the  scalar  quadratic  control  effort  (see  Equa¬ 
tion  (3.3.1)).  To  generate  this  metric  an  optimal  control  policy  ( u(t ))  that  perfectly  connects  the  boundary 
states  is  calculated  and  that  control  policy  is  integrated  as  shown  in  Equation  (3.3.1)  to  yield  the  associated 
control  distance  metric.  Control  distance  metrics  discriminate  based  on  how  states  are  connected  by  con¬ 
trol  effort  rather  than  state  distance.  This  means  if  it  is  much  more  expensive  (from  a  control  standpoint)  to 
move  in  one  direction  than  another,  a  control  distance  metric  will  reflect  this  -  a  state  distance  metric  will  not. 
Singh,  Horwood,  and  Poore  [245]  adapted  the  Control  Distance  Metric  approach  by  using  a  minimum-fuel 
cost  function,  which  yields  impulsive  control  policies  rather  than  smooth  continuous  controls.  This  makes 
the  method  far  more  numerical  than  the  quadratic  control  policy  approach.  This  optimal  control  framework 
is  shown  to  be  quite  effective  at  maneuver  detection  and  characterization  in  data-sparse  systems.  Also,  it 
only  requires  two  state  estimates  at  different  epoch  that  may  be  generated  by  an  independent  estimation  pro¬ 
cess,  so  it  does  not  require  cooperative  observation.  This  satisfies  all  of  the  requirements  of  the  introduced 
problem,  so  it  is  a  good  framework  from  which  to  approach  this  problem. 


dc  =  f  u(r)dT  (3.3.1) 

Work  with  this  control  distance  metric  framework  has  been  limited  to  object  correlation,  maneuver 
detection,  and  characterization. 

3.3.2  Approaches  to  Maneuver  Detection 

There  has  been  additional  recent  work  on  maneuver  detection.  We  call  out  two  specific  approaches  that 
take  differing  philosophies  to  this  problem.  One  approach  develops  a  generic  model  for  any  mis-modeling 
or  thrusting  maneuver  between  measurement  epochs,  providing  a  filter  with  an  additional  set  of  parameters 
that  can  represent  a  maneuver  over  an  arbitrary  time  span  and  across  a  wide  range  of  magnitudes  and  maneu¬ 
ver  morphologies.  The  other  approach  utilizes  the  control  distance  metric  defined  above  to  systematically 
separate  non-gravitational  modeling  errors  from  what  may  be  maneuvers.  Both  of  these  approaches  rely  on 
and  are  aided  by  the  inclusion  of  realistic  covariances.  Further,  when  implemented  systematically  both  of 
these  methods  will  enable  the  formal  or  consider  covariances  for  an  object  to  align  more  strongly  with  a 
realistic  covariance. 
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3.3.2. 1  Essential  Thrust  Coefficients 

In  an  earlier  series  of  research  by  Hudson  and  Scheeres  [129-131]  a  particular  averaging  result  regarding  the 
Gauss  equations  using  arbitrary  thrusting  coefficients  was  discovered  and  investigated.  That  work  focused 
on  modeling  arbitrary,  infinite  dimensional  control  sequences  using  Fourier  Series  expansions  in  different 
orbital  anomalies.  It  was  found  that  by  expanding  an  arbitrary  control  as  a  Fourier  Series  in  Eccentric 
Anomaly  that  it  could  be  described  by  only  14  coefficients,  proscribing  constant,  period  1  or  period  2  thrust¬ 
ing  in  the  radial,  transverse  or  normal  direction.  The  Hudson  and  Scheeres  papers  [129-131]  investigate  this 
representation  analytically  and  numerically  to  fully  confirm  this  result,  to  explore  the  limitations  when  the 
control  magnitude  is  large,  and  to  investigate  possible  applications  of  this  result. 

More  recently,  Ko  and  Scheeres  [164,  165]  investigated  specific  SSA  applications  of  this  result.  The 
fundamental  idea  behind  this  work  is  the  use  of  these  thrust  coefficients  to  act  as  a  rigorous  basis  with 
which  unobserved  maneuvers  performed  by  a  satellite  could  be  represented.  Specifically,  if  a  non-zero 
control  distance  for  a  satellite  is  detected  (assuming  knowledge  of  the  state  at  two  epochs)  it  should  be 
possible  to  represent  the  unobserved  maneuver  (or  unmodeled  forces)  in  an  infinite  number  of  ways  using  the 
Thrust  Fourier  Coefficients,  as  there  are  only  6  constraints  (if  a  change  in  state  is  detected)  but  14  different 
coefficients  to  be  specified.  To  investigate  this  in  more  detail  a  systematic  study  was  made  to  map  out  the  null 
space  in  the  averaged  mapping  between  Thrust  Fourier  Coefficients  and  changes  in  orbit  elements,  as  this  is 
a  linear  mapping  under  standard  orbit  averaging  assumptions.  There  are  a  number  of  obvious  singularities 
between  these  items,  however  once  those  have  been  removed  there  are  still  many  possible  combinations.  It  is 
important  to  note,  however,  that  1-1  maps  between  Thrust  Fourier  Coefficients  and  changes  in  orbit  elements 
can  be  identified.  In  [165]  there  arc  6  specific  collections  of  coefficients  that  are  identified  as  both  being 
1-1  with  arbitrary  changes  in  orbit  elements  and  providing  good  agreement  between  averaged  predictions 
and  full  numerical  integrations  of  these  thrust  coefficients.  Out  of  this  collection  we  chose  one  particular  set 
using  somewhat  arbitrary  criterion.  This  set  is  notable  as  it  only  involves  constant  and  period  1  coefficients, 
specifically  a  constant  radial  coefficient,  a  constant  and  period  1  cosine  and  sine  transverse  coefficients,  and 
a  period  1  cosine  and  sine  out  of  plane  coefficients. 

The  implication  of  this  result  is  fundamental:  any  maneuver  performed  by  a  satellite  transitioning  be¬ 
tween  two  arbitrary  sets  of  orbit  elements  can  be  represented  on  average  as  an  equivalent  maneuver  involving 
constant  and  period  1  Thrust  Fourier  Coefficients  in  the  radial,  transverse  and  normal  direction.  The  initially 
derived  result  was  just  relevant  for  orbit  elements  under  an  assumption  that  the  thrust  magnitude  is  relatively 
small.  However,  this  representation  can  be  extended  to  non-averaged  thrusting  between  Cartesian  positions 
and  velocities,  indicating  a  level  of  universality  in  these  thrust  coefficients. 

This  realization  is  developed  in  [165]  and  a  computational  algorithm  for  representing  any  change  in  orbit 
state  as  an  equivalent  maneuver  using  a  combination  of  constant  and  period  1  thrust  laws  is  given.  As  such, 
given  a  non-ballistic  trajectory  between  two  epochs  due  to  a  maneuver  (or  due  to  mismodeling)  it  is  possible 
to  smoothly  interpolate  between  these  states  using  a  unique  control  law.  The  procedure  developed  can  be 
carried  out  in  three  basic  steps,  although  the  initial  steps  can  sometimes  be  discarded. 

There  are  several  possible  applications  of  this  result.  In  its  most  basic  form,  the  result  provides  a  unique 
and  rigorous  way  to  interpolate  between  two  non-ballistic  states  at  specified  epochs.  More  recent  studies 
show  that  it  is  possible  to  carry  along  initial  uncertainties  (such  as  covariance  matrices)  along  the  interpolated 
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trajectory  [164].  This  could  enable  non-ballistic  events  along  a  satellite’s  trajectory  to  be  automatically 
incorporated  if  the  start  and  end  orbit  states  are  known,  essentially  enabling  filtering  across  a  maneuver.  A 
significant  benefit  of  this  is  that  it  would  allow  pre  and  post  maneuver  tracking  to  be  combined  into  a  single 
trajectory.  It  also  enables  a  common  representation  of  maneuver  events  using  these  coefficients  instead  of 
estimates  or  guesses  of  what  the  actual  maneuver  was.  Once  the  maneuver  is  expressed  in  this  basis,  it  can 
be  further  analyzed  off-line.  As  one  example  of  this,  mapping  a  particular  set  of  coefficients  into  estimates 
of  a  series  of  impulsive  maneuvers  was  investigated  by  Hudson  and  Scheeres  in  [130]. 

3.3.2.2  Control  Distance  Metric  Decomposition 

Previous  research  has  outlined  how  the  control  distance  metric  can  be  used  to  identify  the  abstract  distance 
between  two  spacecraft  states  in  terms  of  the  optimal  control  guidance  law  to  connect  these  states  and  epoch 
[120].  An  optimal  guidance  law  is  unique  for  the  given  cost  function,  however  the  nature  of  the  guidance 
laws  can  vary  significantly  as  a  function  of  the  cost  function.  As  a  simple  example,  a  minimum  norm 
cost  function  will  generally  result  in  discontinuous  maximum  thrust  segments,  whereas  a  least-squares  cost 
function  results  in  a  continuous  and  smooth  thrusting  profile  that  is  always  on. 

More  fundamentally,  the  existence  of  a  non-zero  control  distance  does  not  mean  that  a  maneuver  has 
occurred,  however,  as  any  modeling  error  in  the  dynamics  or  errors  in  the  boundary  states  would  also  result 
in  a  non-zero  control  distance  between  two  disparate  states.  This  situation  raises  a  fundamental  question 
regarding  the  utility  of  the  control  distance  as  a  maneuver  detection  methodology. 

Given  two  states  with  a  non-zero  control  distance,  is  it  possible  to  discriminate  between  modeling  errors, 
state  errors  or  the  existence  of  a  maneuver  between  these  state  epochs? 

A  methodology  has  been  developed,  based  on  the  control  distance  theory,  to  discriminate  between  these 
competing  possibilities.  A  detailed  study  of  this  method  is  presented  and  outlined  in  [181,  182].  The 
method  is  motivated  by  a  few  observations  regarding  optimal  guidance  laws,  mis-modeling  and  environ¬ 
mental  forces.  As  this  method  attempts  to  address  questions  for  which  there  is  insufficient  information, 
there  is  no  definitive  answer  to  the  question  regarding  whether  a  maneuver  has  occurred.  However,  the 
approach  identifies  common  mis-modeling  hypotheses  and  applies  them  to  determine  if  control  distance 
discrepancies  still  exist  once  gross  modeling  errors  arc  corrected. 

Observations  on  environmentally  mis-modeled  forces  Environmental  forces  that  act  on  space  objects 
are  generally  continuous  and  smooth.  Exceptions  occur  if  an  abrupt  change  in  the  space  object  properties 
occur,  however  this  would  nominally  be  labeled  as  a  maneuver.  Furthermore,  the  nature  and  magnitude  of 
the  forces  that  act  on  space  objects  are  quite  well  known,  with  some  specific  exceptions.  Thus,  the  gravity 
field,  3rd  body  perturbations,  and  even  upper-atmosphere  density  errors  are  either  small  or  can  be  tracked 
and  calibrated  after  the  fact. 

A  major  exception  to  this  are  the  non-gravitational  parameters  that  describe  how  the  environmental 
forces  of  atmospheric  drag  and  solar  radiation  pressure  act  on  a  space  object.  Even  though  the  environmental 
component  of  these  forces  can  be  well  characterized  or  reconstructed,  the  relevant  coefficients  can  and  in 
general  do  have  significant  errors  and  uncertainties.  Due  to  this,  the  prime  and  most  common  errors  in  the 
force  models  acting  on  a  space  object  are  due  to  errors  in  these  coefficients.  The  method  described  here 
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directly  relies  on  this  fact  and  focuses  on  improvement  of  these  coefficients  as  the  prime  pathway  to  model 
improvement.  In  the  following  discussion,  and  in  the  theory  as  outlined  in  [181,  182],  we  assume  a  simple 
cannonball  model  for  the  object  in  question.  We  note  that  this  could  be  replaced  with  a  higher  fidelity  model 
if  additional  information  on  an  object’s  geometry  or  attitude  is  known. 

Observations  on  control  distance  cost  metrics  As  noted  above,  the  cost  metric  used  in  computing  the 
control  distance  has  a  fundamental  influence  on  the  properties  of  the  resulting  optimal  guidance  law.  In 
a  detailed  study  provided  in  [182]  and  summarized  in  [181],  it  is  found  that  a  least-squares  cost  metric 
will  mimic  un-modeled  drag  and  SRP  forces.  This  is  not  wholly  unexpected,  as  a  least-squares  optimal 
guidance  law  will  be  continuous  and  smooth  and  will  strive  to  minimize  the  overall  deviation  of  the  control 
acceleration  from  zero.  Similarly,  while  natural  forces  are  not  in  any  sense  optimal,  the  motion  that  ensues 
from  physical  forces  does  satisfy  certain  optimality  constraints.  Specifically,  the  trajectory  arising  from 
applied  forces  will  minimize  the  action  integral,  which  in  general  will  have  a  quadratic  structure  in  the 
vicinity  of  the  true  trajectory.  In  contrast,  a  minimum  norm  cost  metric  would  mimic  natural  forces  with  a 
finite  series  of  impulses,  which  is  far  from  then-  physical  realization. 

Method  for  developing  modeling  updates  The  above  observations  motivate  the  following  methodology 
for  identifying  and  correcting  modeling  errors  that  would  otherwise  be  ascribed  to  state  errors  or  maneuvers. 
The  process  given  in  [181]  is  as  follows: 

1 .  Compute  the  control  distance  and  associated  guidance  law  for  a  least-squares  cost  function 

2.  Fit  atmospheric  drag  and  SRP  forces  against  this  guidance  law,  minimizing  the  difference  between 
the  optimal  guidance  law  and  the  physics-based  force  functions  by  choosing  updated  coefficients 

3.  Apply  estimated  coefficient  corrections  to  existing  non-gravitational  coefficients 

4.  Recompute  control  distance  and  optimal  guidance  law  with  new  non-gravitational  force  terms 

5.  Repeat  until  convergence  is  achieved,  represented  by  zero  or  minimal  updates  to  non-gravitational 
coefficients 

Following  this  process,  the  resulting  control  distance  and  optimal  guidance  law  can  be  subject  to  further 
inspection  and  analysis  to  determine  if  any  non-zero  components  are  due  either  to  state  errors  or  to  the 
existence  of  a  maneuver.  If  the  non- zero  control  distance  was  due  entirely  to  non-gravitational  coefficient 
mis-modeling  this  process  could  recover  the  true  model  and  yield,  in  the  limit,  a  zero  control  distance. 

This  process  is  separate  from  a  traditional  filter  estimation  in  a  few  ways.  First,  in  the  control  distance 
computation  part  there  are  no  hypotheses  on  what  the  non-zero  control  distance  is  due  to.  This  ensures 
that  the  generated  optimal  guidance  law  is  purely  a  function  of  the  current  physics  and  state  mis-match. 
Second,  in  the  update  phase,  the  solution  that  minimizes  the  difference  between  the  optimal  guidance  law 
and  the  non-gravitational  physics  has  a  simple  and  immediate  solution,  requiring  no  further  iteration  for 
the  given  guidance  law.  Finally,  this  process  does  not  try  to  remove  all  state  offsets  by  model  adjustment, 
but  will  only  remove  those  portions  that  can  reasonably  be  removed  through  improved  non-gravitational 
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coefficients.  References  [181,  182]  provide  significant  details  and  examples  of  how  this  methodology  can 
be  applied  to  Earth  orbiters,  incorporating  non-simplistic  models  for  environmental  forces. 

3.4  Sensor  Tasking  and  Scheduling 

The  goal  of  sensor  tasking  and  scheduling  for  a  network  of  sensors  is  to  allocate  resources  appropriately  in 
order  to  gain  as  much  information  as  possible  about  a  system.  A  sensor  tasker  and  scheduler  must  optimize 
system  performance  while  simultaneously  meeting  as  many,  if  not  all,  of  the  requirements  as  possible. 
Sensor  tasking  and  scheduling  also  entails  re -planning  those  tasks  that  cannot  be  completed  over  a  specified 
time  period.  While  the  architectures  for  planning  the  assignment  of  tasks  to  sensors  over  a  single  or  multiple 
time  periods  may  be  centralized  or  distributed/decentralized,  the  function  of  scheduling  is  normally  left  to 
the  sensor  level  processing  as  opposed  to  the  network  level. 

An  excellent  summary  of  the  current  system  used  by  the  United  States  to  perform  the  detection  and 
tracking  functions  of  SSA,  the  Space  Surveillance  Network  (SSN),  is  contained  in  [202],  which  includes 
current  methods  for  tasking  the  network  as  well  as  proposed  improvements.  Other  references  documenting 
sensor  management  techniques  in  applications  relevant  to  the  SSA  mission  include  Sharma  et.  al.  [240], 
which  discusses  the  tasking  of  a  space-based  space  surveillance  sensor,  and  Ben-Asher  &  Cohen,  which 
covers  scheduling  of  radar  resources  in  the  context  of  a  Ballistic  Missile  Defense  (BMD)  scenario.  [30]. 

Of  specific  importance  is  the  distinction  between  sensor  tasking  and  sensor  scheduling.  Sensor  tasking 
involves  the  generation  of  a  set  of  tasks  that  a  sensor  or  sensor  network  is  intended  to  accomplish,  leaving 
the  specifics  of  accomplishing  those  tasks  (e.g.,  timing,  etc.)  to  the  performing  entities.  By  contrast,  sensor 
scheduling  involves  not  only  the  assignment  of  tasks,  but  also  the  decisions  on  the  specifics  involved  in  the 
tasks,  to  include  the  timing  of  when  the  task  must  be  accomplished. 

This  distinction  is  important  due  to  the  current  state  of  affairs  of  the  US  Space  Surveillance  Network 
and  the  associated  commanding  and  tasking  systems,  which  were  designed  around  a  centralized  tasking  and 
distributed/decentralized  scheduling  architecture.  Beyond  the  issues  of  net-centric  command  and  control 
infrastructure,  there  are  also  further  complications  in  the  implementation  of  centralized  scheduling  of  some 
of  the  SSN  sensors  due  to  their  primary  mission  as  missile  warning  systems.  Given  that  RSO  tracking  is 
a  secondary  mission  for  these  radars,  it  is  unclear  how  this  constraint  would  be  posed  or  handled  within 
centralized  sensor  scheduling  approaches.  Sensors  also  have  constraints  in  regards  to  the  number  of  tracks 
that  can  be  collected  per  day  as  well  as  their  ability  to  respond  to  an  arbitrary  sequence  of  tasks.  These 
issues  complicate  algorithms  intended  to  be  implemented  on  the  current  system. 

For  the  rest  of  this  discussion,  we  will  not  make  the  distinction  between  tasking  and  scheduling,  but  will 
instead  refer  to  whichever  is  appropriate  via  the  term  sensor  management,  which  has  been  adopted  widely 
in  the  literature  [211,  225].  In  general,  sensor  management  has  been  posed  as  an  optimization  problem 
under  uncertainty  whether  it  be  a  stochastic  dynamic  programming  or  deterministic.  In  the  case  of  acquiring 
sensor  reports  or  measurements  of  orbits,  one  has  to  optimize  the  use  of  space  track  sensors,  radars  for  low 
altitude  and  telescopes  for  high  altitude  objects,  in  the  collection  of  data.  In  essence,  the  optimal  control 
problem  is  to  pick  an  assignment  of  resources  (sensor  observation  time)  against  tasks  (collection  of  data 
on  various  space  objects)  that  minimizes  some  cost  or  utility  function,  subject  to  various  constraints  (e.g., 
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line-of-site  exists  between  sensor  and  target,  etc.).  The  resulting  problem  is  variously  known  in  the  literature 
as  Information-theoretic  Control/  Active  Sensing  and  Dual  Control  [93,  104,  109,  119]. 

It  is  very  well  known  that  stochastic  control  problems  with  sensing  uncertainty,  of  which  sensor  manage¬ 
ment  problems  are  a  special  case,  can  be  posed  as  a  Markov  Decision  Problem  (MDP)  on  the  information 
state,  which  is  usually  the  conditional  filtered  PDF  of  the  state  [34,  159,  171].  Unfortunately,  such  problems 
are  notoriously  difficult  to  solve  owing  to  the  twin  curses  of  dimensionality  and  history  and  have  only  been 
solved  for  small  to  moderate  sized  discrete  state  space  problems.  A  number  of  approaches  have  been  pro¬ 
posed  using  simulation-based  stochastic  search  techniques  in  an  attempt  to  break  these  computational  issues 
for  moderate  to  large-scale  problems  [118,  135,  251,  252]. 

The  cost  or  utility  function  used  in  these  approaches  can  vary,  but  in  many  cases  it  will  be  some  function 
of  information  gain,  with  information  here  being  the  inverse  of  uncertainty.  In  this  context,  information  gain 
is,  roughly  speaking,  the  reduction  in  uncertainty  that  an  decision  or  sequence  of  decisions  would  yield.  In 
the  context  of  SSA,  this  would  translate  into  the  reduction  in  the  uncertainty  of  the  state  of  the  space  objects, 
with  state  here  including  not  only  typical  track  variables  (position/velocity,  or  some  equivalent  orbit  element 
set),  but  also  possibly  other  parameters  inferable  from  observation  data  such  as  shape,  mass,  area,  etc. 

Many  such  measures  have  been  proposed,  including  simple  covariance  measures  [85,  117]  as  well  as 
more  general  information  divergence  measures  including  Kullback-Leibler  [113,  168],  Renyi  [167,  226], 
and  Cauchy-Schwartz  [66].  Regardless  of  what  measure  is  used,  the  gain  is  calculated  between  the  ex¬ 
pected  uncertainty  that  would  result  if  a  sensor  observation  was  commanded  (an  expected  posterior)  and 
the  expected  uncertainty  if  one  was  not  (the  prior).  The  problem  is  then  to  find  the  sensor-object  pairings 
that  maximize  the  aggregate  or  sum  of  this  information  gain  over  all  possible  sensor-object  pairings.  This 
optimal  pairing  becomes  the  optimal  management  approach,  and  determines  the  future  observations. 

This  feedback  between  the  result  of  the  observations  and  what  observations  will  be  collected  in  the 
future  couples  the  particulars  of  the  estimation  approach  used  and  the  management  approach  used.  If  the 
uncertainty  in  the  predicted  state  is  not  realistic,  some  objects  will  receive  un-needed  updates  that  don’t  sig¬ 
nificantly  reduce  our  uncertainty,  while  and  others  may  be  lost  due  to  lack  of  observations  that  were  actually 
needed.  Thus,  the  need  for  consistent  covariances  and  more  generally  realistic  uncertainty  permeates  the 
management  of  the  sensor  resources. 

Research  into  the  effects  of  this  coupling  has  provided  a  number  of  interesting  results  -  specifically,  that 
the  feedback  between  the  estimation  and  tasking/scheduling  results  in  cases  where  improving  the  estimation 
method  only  (and  keeping  the  management  method  the  same)  results  in  less  data  being  required  to  achieve 
lower  uncertainties  [288]. 

Variations  on  the  general  optimization  approach  discussed  above  have  been  examined  in  the  literature. 
Investigations  have  been  made  into  how  to  use  sensor  management  to  drive  the  shape  of  the  uncertainty  to 
desired  forms  [160]  rather  than  simply  reducing  the  scalar  information  gain  during  the  optimization  process. 
Other  measures  beyond  information  gain  have  been  considered  for  use  in  optimizing  sensor  tasking/schedul¬ 
ing  -  for  example,  the  use  of  Lyapunov  exponents  was  considered  in  [289].  This  approach  is  interesting  in 
that  it  focuses  observations  on  those  objects  with  dynamics  that  arc  the  least  stable  with  respect  to  the  mod¬ 
eled  dynamics  (i.e.,  the  objects  hardest  to  predict  accurately).  This  approach  has  considerable  connections 
to  Orbit  Classes  used  in  [202]. 
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Beyond  simple  metric  observations,  the  SSN  system  is  also  tasked  within  the  space  community  to  collect 
other  types  of  observations  that  may  require  operating  in  a  different  mode  or  involve  different  constraints. 
Examples  of  this  would  be  the  operation  of  a  sensor  in  different  track  modes  in  order  to  collect  astrometric 
data  (e.g.,  right  ascension  &  declination  measurements)  versus  photometry  (light  curve)  data.  Another 
example  would  be  the  need  to  operate  a  sensor  in  a  search  mode  to  search  for  new,  as  yet  undetected, 
objects  versus  a  track  mode  in  order  to  provide  additional  data  on  objects  that  have  already  been  detected. 
Research  has  been  conducted  on  the  development  of  methods  that  can  deal  with  hybrid-state  (discrete¬ 
valued  elements  and  continuous-valued  elements)  estimation  and  sensor  management  problems  that  arise  in 
these  situations.  An  example  hybrid  state  would  include  a  quantized  element  related  to  how  many  objects 
have  been  detected  combined  with  estimated  state  variables  (position/velocities)  for  these  objects.  Random 
Set  Theory  and  the  Finite  State  Statistics  (FISST)  approaches  provide  an  elegant  and  theoretically  rigorous 
approach  to  such  problems,  but  have  the  issue  of  high  computational  requirements  and  poor  scaling  with 
problem  size.  Research  has  been  conducted  combining  the  FISST  framework  with  computationally  efficient 
approaches  for  uncertainty  propagation  that  show  some  promise  to  providing  a  unified  framework  that  can 
handle  hybrid-state  estimation  and  management  tasks  [79-82,  134,  135].  But  the  computational  complexity 
of  these  approaches  is  still  daunting  when  faced  with  the  thousands  of  objects  that  the  SSN  must  track. 

In  view  of  the  current  SSN  system,  algorithms  and  methodologies  that  can  deal  with  the  distributed  de¬ 
cision  making  by  the  individual  sensor  sites,  partial  (delayed)  information  due  to  the  human-driven  tasking 
cycle  timing,  and  uncertain  external  constraints  (such  as  those  represented  by  the  primary  missile  warn¬ 
ing  mission)  are  of  high  interest.  Approaches  could  include  a  game-theoretic  approach,  where  the  sensor 
sites  that  will  implement  the  tasking  are  independent  agents  not  under  the  direct  control  of  the  centralized 
planning  agent.  Such  approaches  would  have  to  consider  all  possible  actions  by  the  distributed  agents  in 
the  development  of  a  schedule/tasking  plan,  not  only  a  single  action  from  each  player  that  is  optimal  with 
respect  to  (for  example)  an  information  utility  function.  The  ability  to  model  the  response  of  sensor  sites  to 
tasking  or  scheduling  commands  would  be  helpful  -  even  a  stochastic  response  model  (e.g.,  faced  with  this 
request,  the  site  will  perform  the  request  with  such-and-such  probability)  would  be  helpful  for  future  efforts. 
Techniques  that  can  include  the  ability  to  handle  task  dependent  constraints  (e.g.,  time  required  to  re -point 
from  last  observation  taken)  and  various  communications  constraints  (e.g.,  space-based  systems  are  not  in 
24/7  ground  contact)  would  also  be  relevant.  Some  theoretical  research  in  distributed  optimal  filtering  under 
communication  constraints  has  been  performed  along  these  lines  in  the  controls  community  driven  by  other 
applications  [214,  216]. 
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Chapter  4 


Structural  and  Parametric  Uncertainties  in 
the  Astrodynamic  Force  Models 


This  chapter  has  three  parts.  Section  4. 1  presents  generic  reference  systems,  time  systems,  and  astrodynamic 
force  models  used  as  a  reference  for  discussing  the  uncertainties  in  these  models.  Section  4.2  makes  an 
attempt  to  delineate  and  quantify  the  structural  and  parametric  uncertainties.  Recommendations  for  reducing 
and  properly  characterizing  these  uncertainties  are  given  in  Section  4.3. 

4.1  Force  Models,  Time,  and  Reference  Systems 

Commonly  used  reference  systems,  time  systems,  and  force  models  for  orbit  determination,  orbit  propaga¬ 
tion,  and  other  SSA  functions  are  presented  in  Subsections  4.1.1,  4.1.2,  and  4.1.3,  respectively.  Much  of  the 
material  is  extracted  with  permission  from  Vallado  [262,  Chapters  8  and  9],  which  was  also  the  basis  for 
Vallado  [261]  and  Gurfil  [106]. 

4.1.1  Reference  Systems 

The  equations  of  motion  are  naturally  expressed  and  solved  in  an  inertial  frame  of  reference.  For  example, 
one  can  use  the  Geocentric  Celestial  Reference  Frame  (GCRF)  as  the  inertial  reference  frame,  and  the 
International  Terrestrial  Reference  Frame  (ITRF)  as  the  Earth-fixed  reference  frame.  Hence,  the  coordinate 
frame  rotations  include  the  International  Astronautical  Union  (IAU)  1976  precession,  IAU  1980  nutation, 
IAU  1982  Greenwich  Mean  Sidereal  Time  (GMST),  and  the  1994  equation  of  equinox*.  This  is  known  as 
the  IAU-76/FK5  system  and  the  transformations  are  detailed  in  International  Earth  Rotation  and  Reference 
Systems  Service  (IERS  TN-13).  Corrections  are  periodically  made  with  respect  to  coordinate  systems  and  a 
series  of  updates  has  resulted  in  the  IAU  2010  conventions  (IERS  2010)  which  make  use  of  the  non-rotating 
origin.  This  is  the  recommended  procedure  of  the  IERS.  The  1996  conventions  were  an  intermediate  form 

*The  inertial-to-fixed  (and  fixed-to-inertial)  rotations  are  used  in  several  places  in  a  tracking  system.  During  orbit  propagation, 
evaluation  of  the  Earth  gravity  model  (Subsection  4. 1.3.1)  requires  these  rotations,  as  do  measurement  transforms  involving  right 
ascension/declination  pairs. 
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that  is  still  sometimes  used  [223].  More  information  on  these  models  may  be  found  in  the  conventions 
descriptions  [207]  and  [262], 

The  celestial  frame  is  related  to  a  time-dependent  terrestrial  frame  through  an  Earth  orientation  model, 
calculated  by  the  standard  matrix-multiplication  sequence  of  transformational  rotations: 

rGCRF  =  P(t)N(f)R(t)W(i)r/TRF  (4.1.1) 

where  vqcrf  is  location  with  respect  to  the  GCRF,  P  and  N  are  the  precession-nutation  matrices  of  date  (t), 
R  is  the  sidereal-rotation  matrix  of  date  (t),  W  is  the  polar-motion  matrix  of  date  (t),  and  tjtrf  is  location 
with  respect  to  the  ITRF.  A  combined  PN  matrix  may  be  formed  as  a  single  operator,  depending  on  the 
theory  adopted.  The  rotations  in  equation  (4.1.1)  are  collectively  known  as  reduction  formulas  (Seidelmann, 
[236])  or  an  Earth  orientation  model.  There  are  two  approaches  for  the  reduction  formulae:  the  classical 
transformation  and  the  Celestial  Intermediate  Origin  (CIO).  The  latter  theory  is  defined  by  IAU  Resolutions. 
The  IAU-2010/2000  Resolutions  were  officially  released  over  a  period  of  time  (McCarthy  and  Petit  [198], 
and  Kaplan  [162]).  These  resolutions  stated  that  beginning  January  1,  2003,  the  IAU- 1976  Precession 
Model  and  the  IAU- 1980  Theory  of  Nutation  were  replaced  by  the  IAU-2000A  Precession-Nutation  theory 
(accurate  to  0.0002”).  The  IAU-2010  Conventions  combined  and  updated  these  resolutions  (Petit  and  Luzum 
[223]) .  As  of  January  1,  2009,  the  IAU-2000A  Precession-nutation  theory  was  separated  into  the  IAU-2000 
Nutation  theory  and  the  IAU-76  Precession  was  replaced  with  the  IAU-2006  Precession,  also  called  the 
P03  model  (Capitaine,  Wallace  and  Chapront[45],  and  Wallace  and  Capitaine  [280]).  Before  2003,  the 
recommended  reduction  to  GCRF  coordinates  was  the  IAU  1980  Theory  of  Nutation,  plus  the  corrections 
(<5A'0i98o>  <5Aei98o)  given  in  the  Earth  Orientation  Parameters  (EOP)  data. 

4.1.2  Time  Systems 

The  transformations  between  the  GCRF,  ITRF,  and  intermediate  systems  are  defined  by  terrestrial  time 
(TT),  which  can  be  determined  directly  from  universal  time  coordinated  (UTC),  the  time  system  in  which 
measurements  are  typically  provided.  Terrestrial  time  is  a  dynamical  time  scale  that  can  be  used  for  precise 
orbit  propagation,  and  when  converting  from  UTC  to  TT,  leap  seconds  are  taken  into  account.  Time  can  be 
expressed  in  seconds  with  respect  to  the  J2000  epoch. 

Solar  time  is  based  on  the  interval  between  successive  transits  of  the  Sun  over  a  local  meridian,  which 
establishes  the  solar  day.  This  concept  has  served  adequately  for  ages.  The  Sun’s  apparent  motion  results 
from  a  combination  of  the  Earth's  rotation  on  its  axis  and  its  annual  orbital  motion  about  the  Sun.  UTC 
is  the  common  time  in  use  today.  Sidereal  time  is  the  time  between  successive  transits  of  the  stars  over 
a  particular  meridian.  Because  the  stars  are  several  orders  of  magnitude  more  distant  than  the  Sun,  their 
relative  locations  as  seen  from  Earth  don’t  change  much  even  during  a  year.  UT1  is  the  common  sidereal 
time  in  use.  The  most  commonly  used  time  system  is  Coordinated  Universal  Time,  UTC,  which  is  derived 
from  an  ensemble  of  atomic  clocks.  It's  designed  to  follow  UT1  within  0.9s  (AUT1  =  UT1  -  UTC).  UTC  is 
the  basis  of  civil  time  systems  and  is  on  ordinary  clocks.  This  definition  of  UTC  was  introduced  in  January, 
1972,  as  a  convenient  approximation  of  UT1.  It’s  sometimes  called  Zulu  time,  but  UTC  is  more  precise. 
Because  UT1  varies  irregularly  due  to  variations  in  the  Earth’s  rotation,  we  must  periodically  insert  leap 
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seconds  into  UTC  to  keep  the  two  time  scales  in  close  agreement. 

A  highly  accurate  time  system  which  is  independent  of  the  average  rotation  of  the  Earth  is  based  on  a 
highly  regular  occurrence  -  International  Atomic  Time,  Temps  Atomique  International,  or  TAI,  is  based  on 
counting  the  cycles  of  a  high-frequency  electrical  circuit  maintained  in  resonance  with  a  cesium- 133  atomic 
transition.  Terrestrial  time,  TT,  is  the  theoretical  timescale  of  apparent  geocentric  ephemerides  of  bodies 
in  the  solar  system  (Seidelmann  [236]).  TT  is  independent  of  equation  of  motion  theories  and  uses  the  SI 
second  as  the  fundamental  interval.  It  is  related  to  other  times  as  follows: 

UTC  =  UT1  -  AUT1 
TAI  =  UTC  +  A  AT 
TT  =  TAI  +  32. 184s 


4.1.3  Force  Models 

This  section  discusses  the  various  generic  force  models  including  Earth  gravity  models,  third-body  pertur¬ 
bations,  drag  models,  solar  radiation  pressure  models,  and  tide  models.  Only  accelerations  on  the  order  of 
10”  1 1  km/s2  and  above  are  considered  (see  Figure  3.1  of  [207]). 

4. 1.3.1  Earth  Gravity  Models 

Modeling  Earth  gravitation  uses  a  spherical  harmonic  potential  equation  in  an  Earth-centered,  Earth-fixed 
reference  frame  with  the  origin  at  the  center  of  mass  of  the  Earth.  The  gravitational  potential  acts  on  a 
satellite  and  is  given  as: 

oo  n 

i+££ 

n=2  ra=0 

where  fi  is  the  gravitational  parameter,  r  is  the  satellite  radius  magnitude,  ipgcSat  and  \sat  are  the  geocentric 
latitude  and  longitude  coordinates  of  the  satellite,  respectively,  is  the  earth  radius,  n  and  m  are  the 
degree  and  order,  respectively,  and  Cnm  and  Snrn  are  the  gravitational  coefficients^.  The  acceleration  on 
the  satellite  is  the  positive  gradient  of  the  potential  function,  and  many  approaches  exist  for  evaluating  said 
gradient  (as  well  as  different  formulations  that  are  singularity-free).  The  Legendre  polynomials  are  defined 
by 

1  dn  fim 

Pn{x)  =  2^\d^{x2~  1)n  and  Pnmix)  =  (1  "  x2rl2d^Pnix)- 

The  Legendre  functions  (polynomial  or  associated  function)  are  called  zonal  harmonics  when  m  =  0, 
sectoral  harmonics  when  n  =  m  ,  and  tesseral  harmonics  when  n  /  m. 

The  first  attempts  to  standardize  models  of  the  Earth’s  gravitational  field  and  the  shape  of  the  Earth  were 
begun  in  1961.  Currently  there  are  several  prevailing  gravitational  models  being  used  within  the  scientific 
community  for  a  variety  of  purposes.  These  models  are  determined  from  a  wide  range  of  measurement 

1  For  a  sufficiently  small  degree  and  order,  the  error  in  the  published  gravitational  coefficients  can  be  safely  neglected  for  most 
applications. 
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types,  satellite  inclinations  and  altitudes  including  surface  gravity  measurements  and  satellite  altimetry  data. 
An  example  earth  gravity  model  is  the  (zero-tide)  2008  Earth  Gravity  Model  [221]. 


4. 1.3.2  Third-Body  Perturbations 

The  presence  of  third  bodies  (e.g.,  Sun,  Moon,  Venus,  Jupiter)  perturb  the  trajectory  of  Earth-orbiting  satel¬ 
lites.  The  Sun  and  Moon  give  rise  to  gravitational  perturbations  on  the  order  of  10-9  km/s2,  Venus  and 
Jupiter  on  the  order  of  10"  11  km/s2.  Hence  Solar  and  Lunar  perturbations  are  appreciable,  whereas  pertur¬ 
bations  arising  from  Venus,  Jupiter,  and  the  other  planets  are  not  relevant  in  most  applications.  These  are 
also  called  n-body  perturbations  acting  on  the  satellite.  The  contributions  are  computed  using  a  point-mass 
equation.  However,  the  Sun  and  Moon  also  include  an  indirect  effect  as  an  interaction  between  a  point-mass 
perturbing  object  and  an  oblate  earth.  Thus  the  third-body  perturbation  includes  both  direct  and  in-direct 
terms  of  point  mass  third-body  perturbations.  Such  gravitational  perturbations  are  modeled  by  considering 
the  acceleration  of  a  satellite  by  a  point  mass, 


body  — 


ri 

'  ®-sat 


\  7  sat- 3 


(4.1.3) 


where  G  is  the  gravitational  constant,  r§.sat  and  r®. 3  are  the  Earth-centered-inertial  position  of  the  satellite 
and  third  body,  respectively,  and  rsat. 3  is  the  position  of  the  third  body  relative  to  the  satellite. 

In  order  to  evaluate  (4.1.3),  one  must  determine  the  position  of  the  third  body  at  a  given  time.  A  number 
of  analytical  models  of  varying  fidelity  have  been  developed  for  such  a  purpose  [207].  Alternatively,  one  may 
utilize  high-precision  ephemerides  from  the  Jet  Propulsion  Laboratory  via  the  Horizons  Online  Ephemeris 
System,  provided  that  interpolation  of  sufficient  fidelity  is  used. 


4. 1.3.3  Aerodynamic  Drag  Models 

Atmospheric  density  can  lead  to  significant  perturbation  effects  for  satellites  below  about  1000  km  altitude, 
but  its  effects  can  be  observed  at  altitudes  up  to  2500  km.  The  acceleration  due  to  aerodynamic  drag  for  the 
often  used  cannonball  model  is 


CD  A  CD  A 

&drag  P~n  l^rell^rel  'Crel'^rel 

2m  2m 

p  Atmospheric  density  at  LEO  altitudes  (recently  reviewed  by  Ernmert  [83])  depends  primarily  on  solar 
ultraviolet  (especially  extreme  ultraviolet)  irradiance,  energy  from  the  solar  wind  and  magnetosphere, 
and  dynamical  forcing  from  the  lower  atmosphere.  These  influences  on  density  act  mainly  through 
changes  in  temperature  and  the  associated  expansion  or  contraction  of  the  upper  atmosphere.  The  solar 
and  magnetospheric  energy  inputs  are  usually  represented  in  models  using  proxies,  most  commonly 
the  F  10.7  solar  radio  flux  for  UV  irradiance  and  a  variety  of  ground-based  geomagnetic  activity  indices 
for  the  magnetospheric  energy  input.  The  use  of  proxies  itself  introduces  uncertainty  into  the  modeled 
density,  and  the  differing  choices  and  applications  of  these  proxies  are  a  source  of  model-to-model 
variation.  Lower  atmospheric  influences  are  currently  neglected  in  operational  aerodynamic  drag 
models.  Density,  which  is  perhaps  the  largest  contribution  to  error  in  LEO  orbit  determination  and 
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prediction,  varies  on  a  broad  range  of  spatial  and  temporal  scales,  many  of  which  are  not  captured 
(and  may  never  be  captured)  by  models. 

c [)  The  coefficient  of  drag  is  related  to  the  shape,  as  well  as  the  satellite  materials,  but  ultimately  it  is  a  dif¬ 
ficult  parameter  to  define.  Gaposchkin  [94]  mentions  that  the  c j>  is  affected  by  a  complex  interaction 
of  reflection,  molecular  content,  attitude,  etc.  It  will  vary,  but  typically  not  very  much  as  the  satellite 
materials  usually  remain  constant.  In  addition,  it  depends  on  the  composition  and  temperature  of  the 
atmosphere 

A  The  cross-sectional  area  changes  constantly  (unless  there  is  precise  attitude  control,  or  the  satellite  is  a 
sphere).  This  variable  can  change  by  a  factor  of  10  or  more  depending  on  the  specific  satellite  configu¬ 
ration.  Macro  models  are  often  used  for  modeling  solar  pressure  accelerations  in  orbit  determination, 
but  seldom  if  ever,  for  atmospheric  density.  There  could  also  be  a  benefit  for  applying  this  technique 
to  atmospheric  density  for  propagation. 

m  The  mass  is  generally  constant,  but  thrusting,  ablation,  etc.,  can  change  this  quantity. 

vrei  The  velocity  relative  to  the  rotating  atmosphere  depends  on  the  accuracy  of  the  a-priori  estimate,  and 
the  results  of  any  differential  correction  processes.  Because  it  is  generally  large,  and  squared,  it 
becomes  a  very  important  factor  in  the  calculation  of  the  acceleration.  \vrei\  =  vrei  is  the  magnitude 
of  the  vector  vrei. 

The  ballistic  coefficient  ( II C  =  m/ (cf)A  j))  is  generally  used  to  combine  the  mass,  area,  and  coefficient 
of  drag  values  together.  It  will  vary,  and  sometimes  by  a  large  factor.  In  orbit  determinations,  it  may 
not  be  best  to  model  the  combined  parameter  because  it  includes  several  other  time-varying  parameters 
that  are  perhaps  better  modeled  separately.  There  can  also  be  lift  and  side  force  components  due  to  the 
atmospheric  effects.  This  is  especially  true  for  reentering  objects.  These  additional  components  act  in-plane 
and  out-of  plane.  They  are  rarely  modeled  but  they  do  contribute  to  uncertainty  in  some  situations.  In  orbit 
determination  applications,  it  may  not  be  best  to  model  the  combined  parameter  because  it  includes  several 
other  time-varying  parameters  that  are  perhaps  better  modeled  separately. 

4.1.3.4  Solar  Radiation  Pressure  Models 

The  force  due  to  solar  radiation  pressure  (SRP)  arises  when  photons  from  the  sun  strike  a  satellite  surface 
and  are  absorbed  (or  reflected  -  specular  and  diffuse)  and  thus  transfer  an  impulse  to  the  satellite.  Unlike 
atmospheric  drag,  the  SRP  force  does  not  vary  with  altitude  and  its  main  effect  is  a  slight  change  in  the 
eccentricity  and  longitude  of  perigee.  The  effect  of  SRP  depends  on  the  satellite  mass  and  surface  area  and 
is  most  notable  for  satellites  with  large  solar  panels  like  communications  satellites  and  Global  Positioning 
Satellites  (GPS),  or  with  objects  that  have  High  Area  to  Mass  Ratios  (HAMR).  SRP  is  often  the  dominant 
perturbation  for  HAMR  objects.  In  cases  of  geodetic  precision  orbits,  complex  models  of  the  exposed 
satellite  surfaces  are  created  often  using  finite-element  computer  codes.  This  is  the  case  with  GPS.  To  find 
the  acceleration  due  to  solar  radiation  pressure, 
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crAs  un  sat— Sun 
& srp  —  PSR  ] 

RR  r  sat— Sun 


(4.1.4) 


PSR  The  incoming  solar  pressure  depends  on  the  time  of  year,  and  the  intensity  of  the  solar  output.  Its 
derived  from  the  incoming  solar  flux  and  values  of  about  1358-1373  W/m 2  are  common. 


cr  The  coefficient  of  reflectivity  indicates  the  absoiptive  and  reflective  properties  of  the  material,  and  thus 
the  susceptibility  of  the  satellite  to  the  effects  of  incoming  solar  radiation.  Note  that  recent  research 
indicates  a  bi-directional  cr  may  be  more  realistic  [285]. 


Asun  The  cross-sectional  area  (with  respect  to  the  Sun)  changes  constantly  (unless  there  is  precise  attitude 
control  or  the  satellite  is  a  sphere).  This  variable  can  change  by  a  factor  of  10  or  more  depending  on 
the  specific  satellite  configuration.  Macro  models  are  often  used  for  geosynchronous  satellites.  This 
area  is  generally  not  the  same  as  the  cross-sectional  area  for  drag.  If  the  satellite  attitude  is  known, 
the  time-varying  cross-sectional  area  may  be  found  at  each  time,  thereby  increasing  the  accuracy  of 
any  propagation. 


m  The  mass  is  generally  constant,  but  thrusting,  ablation,  etc.,  can  change  this  quantity. 


f  sat— Sun  The  orientation  of  the  force  depends  on  the  satellite-Sun  vector  and  is  usually  different  from 
atmospheric  drag. 


Similar  to  atmospheric  drag,  the  solar  radiation  force  is  not  necessarily  exactly  along  the  Sun-satellite 
vector.  In  this  case,  the  components  are  related  to  the  ecliptic  plane.  There  are  components  orthogonal  to 
this  vector,  and  they  can  influence  the  uncertainty. 

Accurate  modeling  of  solar  radiation  pressure  is  challenging  for  several  reasons.  A  significant  factor  is 
the  use  of  a  model  to  include  times  when  the  satellite  passes  through  the  Earth’s  shadow.  The  model  may 
take  several  forms,  including  a  simple  cylindrical  model,  to  one  that  models  precise  umbral  and  penumbral 
regions.  Models  also  consider  the  attenuation  effect  that  the  Earth’s  atmosphere  has  on  the  shadow  region. 
Some  programs  accomplish  this  with  an  effective  Earth  radius  that  is  slightly  larger  (say  23  km)  than  the 
Earth’s  physical  size.  Depending  on  the  length  of  time  in  the  shadow,  these  options  can  have  a  dramatic 
effect  on  propagations. 

Determining  the  exact  times  of  entry  and  exit  from  the  Earth's  shadow  is  important  as  the  satellite  will 
usually  cross  from  sunlight  to  being  in  the  shadow  in  the  middle  of  a  numerical  integration  step.  This  usually 
requires  some  type  of  iteration  to  obtain  the  precise  entry  and  exit  times. 

Using  a  single  value  for  the  incoming  solar  luminosity,  or  equivalent  flux  at  1  AU  can  also  alter  the 
resulting  perturbing  values.  There  are  seasonal  variations  in  the  amount  of  incoming  luminosity  that  can  be 
included  in  these  values. 

Albedo  is  the  radiation  pressure  emitted  from  the  Earth  which  causes  a  small  perturbing  force  on  a 
satellite.  Although  the  effect  of  SRP  is  usually  far  larger,  the  effects  of  Earth’s  albedo  can  be  comparable 
for  certain  configurations  of  orbits  (e.g.  sun-synchronous).  The  acceleration  due  to  albedo  is  generally 
expressed  in  terms  of  a  second  degree  zonal  spherical  harmonic  model,  and  contributions  from  various 
Earth  sectors  are  summed  to  determine  the  overall  effect. 
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4. 1.3.5  Tide  Models 

Earth  tidal  effects  on  satellites  are  due  to  pole  tides,  ocean  tides,  and  solid  earth  tides.  Most  of  the  data  that 
have  resulted  in  a  definitive  model  have  come  about  within  the  last  couple  of  decades  from  Earth  observation 
satellites  such  as  TOPEX  and  GRACE.  As  these  satellite  observations  have  been  processed  over  the  last  20 
years,  the  former  rigid  Earth  model  has  slowly  changed  to  an  elastic  Earth  model.  The  basis  of  the  models 
for  pole,  solid  earth  and  ocean  tide  models  can  be  found  in  the  IERS  Conventions,  with  updates  in  McCarthy 
and  Petit  [196-198].  Tidal  models  do  not  enjoy  the  variety  of  the  gravitational  and  atmospheric  models  yet, 
but  there  are  several  different  approaches.  Properly  implementing  these  various  models  can  be  a  factor  if 
precise  comparisons  are  desired.  At  this  point  in  time,  several  models  exist,  and  no  clear  “leader”  has  been 
recognized  as  the  standard  approach. 

Pole  tides  define  the  rotational  deformation  of  the  pole  due  to  an  elastic  Earth.  These  are  modeled  by  the 
C21  and  S21  coefficients  in  the  Earth's  potential.  Solid  earth  tidal  contributions  are  computed  as  corrections 
to  the  spherical  harmonics  coefficients,  as  are  the  ocean  tidal  contributions. 

4.1.3.6  Other  Forces 

As  the  accuracy  of  orbit  determination  and  propagation  increases,  additional  force  models  are  included  in 
analyses.  In  particular,  applications  using  GPS  data  often  must  account  for  the  (primarily)  apsidal  rotation 
caused  by  General  Relativity.  GPS  signals  must  also  be  corrected  for  General  Relativity,  as  well  as  atomic 
clock  corrections.  The  effects  of  General  Relativity  are  very  small  and  only  become  important  where  orbit 
precision  below  the  cm-level  is  needed.  Satellite  thrusting  can  also  be  a  significant  perturbing  force.  Many 
satellites  use  maneuvers  for  mission  operation  and  for  orbit  maintenance.  The  forces  induced  by  these  motor 
firings  can  be  large  or  small.  We  do  not  describe  these  in  any  detail,  but  introduce  the  forces  as  something 
needing  to  be  considered  in  mission  planning  and  precision  orbit  determination  modeling. 

4.2  Uncertainties  in  the  Dynamics  and  Space  Environment 

Given  the  above  models  or  any  other  collection  of  models,  is  it  possible  to  characterize  the  structural  and 
parametric  uncertainties?  The  answer  is  yes,  but  there  are  several  considerations  that  are  necessary.  Looking 
at  Figure  3. 1  in  Montenbruck  and  Gill  [207],  the  force  of  atmospheric  drag  is  the  largest  source  of  uncertainty 
in  the  prediction  of  trajectories  of  most  objects  in  low  Earth  orbits,  given  that  uncertainty  in  the  ballistic 
coefficient  can  be  as  high  as  a  few  percent.  Solar  variability  is  the  largest  source  of  error  in  upper  atmospheric 
density  forecasts.  In  addition  to  uncertainties  in  the  force  models,  time  and  coordinate  systems  can  introduce 
uncertainty  in  the  solution. 

EOP  files  are  necessary  for  accurate  time  and  coordinate  system  use  in  orbit  determination  and  propa¬ 
gation,  and  sources  like  CelesTrak  consolidate  current  values.  However,  not  all  systems  use  current  EOP 
files,  and  the  predicted  values  change  continuously,  although  they  are  reasonably  well-behaved.  Bradley, 
Vallado,  Sibois  and  Axelrad  [41]  discusses  the  need  to  interpolate  EOP  values  in  the  same  way  atmospheric 
drag  indices  are  interpolated  (Vallado  and  Finkleman  [264]).  To  assess  the  impact  of  uncertainties  in  time, 
consider  circular  satellites  at  altitudes  of  800  km  (LEO)  and  35780  km  (GEO).  Sensitivity  tests  can  be  run  to 
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determine  the  effect  of  various  methods  employed,  or  envisioned  in  operational  systems  (see  Table  4.2.1).  In 
each  case,  the  same  initial  state  vectors  were  transformed  from  Earth  fixed  (ITRF)  to  Earth  Inertial  (GCRF) 
coordinates,  permitting  a  positional  difference  determination  at  the  end  state. 

The  first  set  of  options  center  mainly  around  the  EOP  values  and  how  they  are  used.  Much  discussion 
has  occurred  with  leap  seconds  (Finkleman,  Seago  and  Seidelmann  [87]).  It  is  possible  that  some  systems 
may  not  use  leap  seconds,  or  may  not  update  them.  This  would  lead  to  errors  of  1  or  2  seconds  in  TAI. 
Because  this  is  used  only  as  an  argument  in  the  transformation,  the  difference  is  quite  small.  If  the  TT  is  off 
by  a  minute,  perhaps  because  of  assumptions  or  EOP  values  simply  not  being  used,  it  is  again  used  as  an 
argument  in  the  transformation. 

The  EOP  files  change  periodically  and  are  updated  daily  to  include  recent  observational  data.  If  old 
EOP  files  are  used,  the  AUTl  value  could  differ  by  0.01  sec.  Here  one  sees  additional  effects  because 
the  UT1  time  is  affected,  along  with  GMST.  If  AUTl  is  not  used,  the  differences  become  larger  in  each 
transformation.  If  the  UTC  is  off  by  a  second,  a  clock  could  be  off,  or  a  script  error  could  be  present.  Since 
UTC  affects  all  the  parameters  and  arguments,  the  differences  are  very  large. 

Finally,  its  conceivable  that  the  time  tags  could  be  off  by  a  second.  This  is  very  different  from  the 
previous  leap  second  and  EOP  errors  because  the  argument  is  no  longer  an  input  into  the  transformation,  but 
rather  a  simple  offset.  The  results  are  significantly  larger.  Notice  that  for  the  case  of  a  1  sec  error  affecting 
the  transformation,  a  vector  closer  to  the  earth  will  move  less  than  a  GEO  satellite.  For  the  case  of  a  time 
tag  error  of  1  sec,  the  satellites  will  be  off  by  the  orbital  velocity  over  that  time  interval. 


Table  4.2.1  EOP  sensitivity  tests:  several  tests  were  run  to  examine  potential  errors  and  the  effect  on  satellite 
positions  from  various  EOP  discrepancies.  The  time  tag  error  is  not  a  transformation  error  and  is  therefore 
larger  than  the  UTC  time  argument  error  of  1  second. 


Test 

Possible  Cause 

LEO 

GEO 

Notes 

TT  is  off  by  a  min 

Not  using  TT 

0.000  m 

0.000  m 

TT  is  an  argument 

AAT  error  of  1  sec 

Old  EOP  file 

0.000  02  m 

0.000  10  m 

Only  TAI  affected 

Correction  sources 
off  by  0.0001  " 

Differences  in  AdX/dY  corrections 

0.035  m 

0.240  m 

AUTl  value  differs 
by  0.01  sec  AUTl 

Truncation,  old  EOP 

6.0  m 

31.0m 

Ignoring  AUTl, 

assume  -  0.25  sec 
AUTl 

Not  using  EOP 

145.0  m 

800.0  m 

UTC  error  of  1  sec 

Clock  off 

581.0  m 

3075.0  m 

UTC  is  an  argument 

Time  tag  is  incorrect 
by  1  sec 

Script  error 

7500.0  m 

3075.0  m 

Bookkeeping  error, 
(velocity  x  time) 

The  force  models  each  contribute  uncertainty  as  they  are  not  perfect,  and  they  all  rely  on  data  that 
has  varying  degrees  of  accuracy.  Studies  generally  examine  either  the  perturbation  accelerations  or  an 
ephemeris  accuracy  compared  to  independent  reference  orbits  such  as  Satellite  Laser  Ranging  derived  orbits. 
Vallado  [259]  examined  the  effect  of  various  perturbations  on  satellite  propagation  as  well  as  introducing 
the  necessary  parameters  to  align  to  ensure  similar  orbit  propagations  when  using  multiple  propagation 
techniques  or  programs.  Everything  else  was  kept  constant,  but  the  gravity,  drag,  solar  radiation  pressure, 
third  body,  etc.,  were  changed  individually  and  compared  to  quantify  how  much  impact  each  perturbation 
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had  on  the  resulting  ephemeris. 

For  gravity,  the  uncertainty  has  been  reduced  substantially  from  the  1960’s  when  different  gravity  field 
models  would  introduce  km-level  differences  in  propagation.  The  fields  have  grown  substantially  over  the 
last  decade.  EGM-96  was  360  x  360,  but  EGM08  is  2190  x  2190!  Thus,  gravity  fields  are  often  truncated  to 
smaller  sizes  to  afford  greater  computational  speed  (36  x  36,  24  x  24  for  instance).  The  increase  in  size  has 
also  driven  many  new  approaches  to  fast  interpolation  of  the  larger  field  sizes  [19,  147,  230], 

Atmospheric  drag  presents  many  additional  challenges,  and  inserts  the  largest  uncertainty  into  the  solu¬ 
tion  for  low-Earth  orbiting  satellites.  Vallado  and  Finkleman  [264]  performed  a  detailed  study  of  many  of 
the  factors  contributing  to  the  uncertainty  resulting  from  atmospheric  drag.  The  analysis  consisted  of  two 
major  sections.  In  the  first,  low-Earth  satellite  orbits  were  propagated  with  a  single  atmospheric  model  and 
data  parameter  implementation  to  form  a  reference  or  baseline  configuration.  Then,  numerous  variations  of 
atmospheric  models  and  data  input  parameters  were  compared  to  this  baseline  ephemeris  to  gauge  the  in¬ 
fluence  of  each  parameter  on  the  propagated  orbits.  The  second  analysis  used  Precision  Orbit  Ephemerides 
(POEs)  as  observations  to  an  Orbit  Determination  (OD)  process.  This  was  an  important  addition  to  the 
simple  propagation  of  the  first  test  because  the  OD  process  “corrects”  many  of  the  discrepancies  in  the  first 
test  and  provides  a  slightly  different  and  potentially  more  accurate  examination  of  the  influence  of  various 
parameters,  and  atmospheric  models.  The  major  error  sources  found  in  using  atmospheric  models  are  listed 
below  and  are  a  combination  from  the  two  analyses  performed  by  Vallado  and  Finkleman  [264]. 

•  Using  predicted  values  of  T10.7,  Kp,  ap  for  real-time  operations 

•  Lack  of  satellite  attitude  in  determining  the  time-varying  cross-sectional  area 

•  Using  different  atmospheric  models 

•  Not  using  the  actual  measurement  time  for  the  values  (F10.7  in  particular  at  2000  UTC) 

•  Using  step  functions  for  the  atmospheric  parameters  vs  interpolation 

•  Using  the  last  81 -day  average  F\  0,7  vs.  the  central  81 -day  average 

•  Using  observed  or  adjusted  space  weather  parameter  values 

•  Using  undocumented  differences  from  the  original  atmospheric  model  technical  definition 

•  Not  accounting  for  (possibly)  known  dynamic  effects  -  changing  attitude,  molecular  interaction  with 
the  satellite  materials,  etc. 

•  Inherent  limitations  of  the  atmospheric  models  in  accurately  determining  spatial  and  temporal  varia¬ 
tions  in  density 

•  Use  of  differing  interpolation  techniques  for  the  atmospheric  parameters 

•  Using  approximations  for  the  satellite  altitude,  solar  position,  etc.  (this  can  be  appreciable  but  is 
difficult  to  address  without  attitude  data  and  knowledge  of  the  shape  and  material  properties) 

•  Using  ap  or  Kp  and  converting  between  these  values 

•  Use  of  -E10.7  vs  F10.7  in  the  atmospheric  models  (this  is  not  well  characterized  yet) 

Solar  radiation  pressure  also  inserts  uncertainty  to  the  solution  and  can  be  significant  for  GEO  and 
HAMR  objects.  Accurate  modeling  of  solar  radiation  pressure  is  challenging  for  several  reasons.  The  major 
error  sources  are: 
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•  Use  of  macro  models/attitude  -  this  is  perhaps  the  largest  difference  between  programs 

•  Use  of  differing  shadow  models  (umbral  /  penumbral  regions,  cylindrical,  none,  etc.) 

•  Using  a  single  value  for  the  incoming  solar  luminosity,  or  equivalent  flux  at  1  AU 

•  Use  of  an  effective  Earth  radius  for  shadow  calculations  (23  km  additional  altitude  is  common)  -  this 
approximates  the  effect  of  attenuation  from  the  atmosphere 

•  Using  different  methods  to  account  for  seasonal  variations  in  the  solar  pressure 

•  Not  integrating  to  the  exact  points  of  arrival  and  departure  at  the  shadow  boundary 

•  Use  of  simplified  treatment  for  the  light-time  travel  from  the  Sun  to  the  satellite  (instantaneous  (true), 
light  delay  to  central  body  accounted  for  (app  to  true),  light  delay  to  satellite  (default)) 

Although  not  as  large  an  effect  as  atmospheric  drag,  the  general  effect  can  be  several  hundred  meters  to  a 
few  kilometers. 

There  is  also  a  lack  of  academic  treatment  of  the  exact  causes  of  the  uncertainty  in  atmospheric  drag 
models  themselves.  There  are  a  few  studies  that  have  begun  to  address  the  problem,  e.g.,  papers  by  An¬ 
derson,  Born,  and  Forbes  [9],  Emmert,  Byers,  Warren,  and  Segerman  [84],  and  Lenard,  Forbes,  and  Born 
[176], 

4.3  Recommendations 

Recall  that  covariance  realism  implies  that  the  estimated  state  be  the  true  state,  and  that  the  estimated  co- 
variance  be  the  true  covariance.  Thus,  while  the  committee  strongly  supports  the  continued  development  of 
improved  models  for  producing  more  accurate  state  estimates,  this  alone  is  not  sufficient  to  achieve  covari¬ 
ance  realism.  A  proper  characterization  of  the  uncertainty  in  each  model  is  needed  so  that  the  uncertainty 
may  be  treated  in  a  statistical  fashion.  For  example,  can  we  model  the  error  in  the  acceleration  vector  due 
to  mismodeled  dynamics  at  a  given  point  in  space  at  a  given  time?  If  done  correctly,  then  any  two  models 
will  produce  solutions  that  agree  to  within  the  level  of  the  uncertainty,  and  it  is  not  necessary  that  the  two 
models  have  the  exact  implementation,  or  even  be  of  the  same  fidelity.  Approaches  for  accounting  for  the 
uncertainties  in  the  models  are  given  in  Section  6.2.  A  recommendation  would  be  to  apply  these  approaches 
to  particular  astrodynamic  models. 

Alternatively  or  in  addition,  by  examination  of  the  structural  and  parametric  uncertainties,  one  can  strive 
to  enable  multiple  organizations  to  arrive  at  propagated  ephemerides  that  compare  to  some  level  of  positional 
accuracy.  We  have  shown  that  there  is  uncertainty  at  each  step  of  process,  from  the  time  and  coordinate 
systems,  and  force  models,  to  the  data  input  parameters  that  drive  the  force  models.  To  achieve  perfect 
alignment  requires  a  very  large  set  of  interface  controls  that  may  be  beyond  a  specific  organization’s  needs. 
Thus,  the  recommendations  are  grouped  into  a  somewhat  descending  order  of  importance  in  each  category 
of  time  and  coordinate  systems,  force  models,  data  input  parameters,  and  parameter  uncertainty  treatment. 

Time  and  coordinate  systems  have  become  complex.  The  latest  IAU-2010  conventions  stipulate  a  pre¬ 
cise  method  of  converting  vectors  between  Earth  fixed  and  inertial  locations,  but  the  full  implementation 
consists  of  literally  thousands  of  mathematical  operations.  Unfortunately,  most  users  still  use  a  variant  of 
the  older  IAU-76/FK5  (which  by  itself  has  a  few  hundred  operations),  or  an  approximation  of  an  intermedi¬ 
ate  form  of  these  systems.  Any  variation  can  lead  to  potentially  large  uncertainties.  Thus,  strict  adherence, 
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use,  and  documentation  of  exactly  which  coordinate  system  is  used  is  imperative  for  any  mutual  operation. 
Likewise,  time  systems  present  many  different  approaches  to  measuring  epochs.  While  UTC  and  GPS  offer 
common  systems  that  are  reasonably  well  understood,  organizations  have  been  known  to  make  assumptions 
where,  for  their  particular  operation,  some  parameters  may  not  be  included.  This  presents  the  potential  for 
step  discontinuities  and  large  positional  differences.  Message  formats  with  insignificant  precision  insert  ad¬ 
ditional  ambiguity  into  measurements  and  observations.  Again,  the  solution  is  to  document  and  understand 
what  the  limitations  of  any  particular  time  system  will  insert  into  the  operation,  and  convey  that  to  any  other 
participating  organization. 

For  the  force  models  themselves,  it  is  imperative  that  the  exact  technical  implementation  is  known 
either  through  detailed  documentation  of  the  equations  and  approach,  or  through  the  availability  of  well- 
documented  computer  code.  Implicit  is  the  understanding  of  any  and  all  assumptions  that  are  used  in  the 
development  and  implementation  of  each  force  model.  Although  gravity  is  by  far  the  most  dominant  per¬ 
turbing  force,  it  is  well  modeled  and  understood  and  is  often  not  a  major  contributor  to  uncertainty.  The 
force  model  uncertainty  effects  for  LEO  satellites  are 

•  Atmospheric  drag, 

•  Solar  Radiation  Pressure, 

•  Earth  Albedo, 

•  Ocean  and  Solid  Tides. 

For  higher  altitude  satellites,  atmospheric  drag  is  not  present  and  therefore  drops  off  the  list.  The  data  input 
parameters  also  can  introduce  uncertainty  into  the  ephemeris  generation  process.  Several  of  these  were 
detailed  in  the  Section  4.2. 

The  single  greatest  improvement  in  atmospheric  drag  and  solar  radiation  pressure  would  be  to  improve 
the  accuracy  of  the  predicted  indices/flux  values.  A  close  second  would  be  to  model  the  dynamic  attitude 
of  the  satellite  so  that  the  individual  cross  sectional  areas  could  be  computed  and  used  with  the  acceleration 
models.  The  remainder  of  the  items  listed  in  the  discussion  represent  second  tier  effects  that  are  important, 
but  have  a  much  smaller  effect  on  the  uncertainty. 

While  the  different  uncertainties  have  been  listed  in  this  section  with  respect  to  size,  the  impact  of 
each  of  these  on  the  propagated  uncertainty  remains  an  open  question,  i.e.,  the  relative  contribution  to  the 
propagated  state  still  needs  to  be  investigated. 
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Chapter  5 


Measurement  Errors  and  Sensor  Level 
Processing 

5.1  Introduction 

Complete  understanding  of  sensor  error  characteristics  and  their  temporal  behavior  is  required  for  covariance 
and  uncertainty  realism.  Observation  noise  and  biases  contribute  to  the  final  covariance  in  different  ways. 
Failure  to  accurately  reflect  sensor  noise  and  biases  correctly  will  result  in  erroneous  covariance  predictions, 
regardless  of  the  quality  of  the  basic  covariance  propagation.  Correct  assessment  and  modeling  requires 
understanding  of  the  intrinsic  sensor  system  observables  and  the  noise  and  bias  behavior  in  that  reference 
system.  The  relative  contribution  of  the  data  errors  and  force  modeling  errors  to  orbit  covariance  is  a  strong 
function  of  the  sensor  performance,  the  object’s  ballistic  coefficients,  and  the  altitude  of  the  object.  It  is 
difficult  to  make  generalisms  -  it  is  best  to  do  a  full  analysis  on  a  case-by-case  basis. 

In  the  past,  sensor  performance  limitations  and  mission  accuracy  requirements  allowed  a  lot  of  approx¬ 
imations  and  short  cuts  to  be  taken  in  the  sensor  data  processing.  As  new,  more  accurate,  sensors  come 
online  these  approximations  can  be  a  significant  source  of  error  in  the  orbit  determination  processing  at  the 
JSpOC.  This  transition  is  similar  to  the  experience  in  the  precision  geodetic  community  as  the  second  and 
third  generation  laser  tracking  systems  and  GPS  came  online. 

This  chapter  of  the  report  is  intended  to  provide  a  generic  background  on  the  intrinsic  observables  and 
sensor  level  processing  that  occurs  in  the  data  reported  to  the  JSpOC  and  used  in  the  catalog  maintenance. 
It  is  hoped  that  this  will  enable  a  more  detailed  assessment  of  the  data.  While  some  of  this  material  appeal's 
basic  to  those  who  work  at  the  sensor  level,  for  many  working  at  the  JSpOC  catalog  processing,  and  general 
orbit  determination  applications,  the  processing  that  occurs  at  the  sensor  is  not  fully  understood  and  is  treated 
as  a  blackbox. 

This  chapter  covers  phased  array  radars,  dish  radars,  hybrid  radars,  and  optical  tracking  systems.  Each 
sensor  type  section  describes  the  processing  of  the  observations  before  they  are  sent  to  the  JSpOC.  For  each 
of  those  sensor  types  the  intrinsic  observables,  generic  sources  of  measurement  biases  and  noise  in  those 
observables,  sensor  level  processing  effects,  and  sensor  level  calibration  are  explained.  Because  of  overlap 
in  functionality,  the  JSpOC  level  calibration  processes  are  also  described. 
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We  also  summarize  the  results  of  the  assessment,  drawing  on  team  experience,  and  make  recommenda¬ 
tions.  Short-term  recommendations  will  focus  on  improvements  at  the  JSpOC  with  currently  available  data. 
It  is  expected  that  sensor  side  improvements  cannot  be  made  in  the  short  term.  Mid-term  recommendations 
focus  on  gaining  better  insight  into  the  actual  sensor  performance  to  assess  the  use  of  the  actual  sensor  de¬ 
tection  data.  Long-term  recommendations  focus  on  changes  to  future  system  requirements  and  algorithms 
to  drive  future  upgrades  and  new  systems  in  the  net-centric  environment. 

5.2  Overview 

Sensor  noise  and  bias  phenomenology  varies  remarkably  between  the  sensor  types.  From  a  modeling  per¬ 
spective,  the  noise  and  biases  can  be  a  function  of  the  sensor  intrinsic  variables,  the  local  topocentric  frame, 
or  celestial  coordinates.  Further,  noise  and  biases  can  behave  differently  at  different  time  scales:  detection- 
to-detection,  track-to-track,  and  long-term  or  constant  behavior,  depending  on  the  sensor  type.  Computation 
of  correct  covariances  ideally  needs  to  reflect  these  time  frames. 

The  data  arc  registered  to  basic  reference  frames.  Radar  data  is  referenced  to  the  Earth  fixed  system, 
through  the  radar  boresite,  and  the  rotation  to  the  local  topocentric  frame.  Optical  data  is  typically  measured 
with  respect  to  the  stellar  background,  and  referenced  to  a  celestial  frame.  The  angular  errors  in  this  kind 
of  optical  data  arc  very  small  compared  to  radar  data.  Some  newer  small  optical  sensors  arc  going  to  use 
Azimuth/Elevation  mounts  and  the  observations  will  be  reference  to  the  local  topocentric  earth  fixed  frame. 
Errors  in  modeling  the  transformation  between  the  earth  fixed  and  celestial  frames  can  result  in  a  bias 
between  the  optical  data  and  radar  data. 

Within  the  SSN,  observations  arc  reported  as  metric  observation  in  the  “B3”  format.  The  B3  format 
supports  several  different  types  of  observations.  For  radars,  the  reported  data  consist  of  a  range,  azimuth, 
elevation  and  range-rate  at  an  indicated  time.  These  are  generally  produced  at  rates  depending  on  the  sensor 
and  conditions.  Right  ascension  and  declination  are  usually  used  for  optical  systems.  While  the  B3  format 
simplifies  data  exchange,  the  limited  options  for  data  representation  can  further  complicate  observation 
processing.  The  intrinsic  observables  arc  not  typically  reported.  Instead,  synthetic  observations,  often 
computed  based  on  track  filters,  arc  reported.  This  can  have  a  significant  effect  on  the  data  errors,  statistical 
behavior,  data  analysis,  and  modeling.  This  is  explained  in  more  detail  in  following  sections. 

Most  of  the  sensors  perform  routine  internal  calibration.  For  phased  array  radars,  precision  reference 
ephemeris  information  from  the  JSpOC,  or  other  sources,  is  used  to  calibrate  the  observed  range  and  angle 
information  generated  by  radar.  This  calibration  data  is  the  stored  in  the  system  to  correct  future  obser¬ 
vations.  This  process  occurs  at  a  schedule  that  is  not  aligned  with  the  JSpOC  calibration  processes.  An 
intense  calibration  process  is  performed  prior  to  system  IOC  or  after  major  refits.  Maintenance,  equipment 
degradation,  or  environmental  effects  can  cause  the  calibrations  to  drift,  triggering  a  recalibration  process. 
As  a  result  the  biases  for  individual  sensor  can  change  during  the  calibration  period  used  by  the  JSpOC  for 
the  entire  network.  This  change  in  the  bias  from  the  computed  value  can  result  in  additional  unaccounted 
for  error  in  the  JSpOC’s  orbit  determination  and  prediction,  reducing  covariance  accuracy. 

Initial  orbit  determination  is  performed  at  most  sites,  but  is  not  usually  reported  to  the  JSpOC.  In  order 
for  the  radar  systems  to  track  uncued  objects,  they  run  a  track  filter  to  determine  where  the  next  track  beam 
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needs  to  be  pointed  (they  do  not  raster  scan  the  entire  sky).  This  track  filter  computes  position  and  velocity 
information  commensurate  with  an  IOD  solution.  In  some  cases  this  is  reported  to  various  customers.  For 
example,  the  Early  Warning  systems  report  the  tracks  for  missile  warning  and  defense  applications.  The 
IOD  reflects  the  best  estimate  of  the  correlation  of  observations  within  a  track  (i.e.  it  is  a  track  of  a  unique 
object).  This  information  is  used  to  generate  and  tag  each  of  the  metric  observations  for  the  track.  The 
metric  observations  are  sent  into  the  JSpOC  as  individual  observations,  losing  the  information  that  all  the 
observations  within  the  track  are  strongly  correlated  to  a  single  object. 

As  sensor  and  catalog  processing  quality  improves,  the  subject  of  sensor  precision  vs  accuracy  will 
become  ever  more  important.  Some  of  the  current  optical  systems,  and  some  of  the  radar  sensors  that  may 
be  used  in  the  future  provide  very  high  quality  observations.  In  general,  the  level  of  noise  in  the  data,  which 
often  drives  precision,  is  decreasing.  This  makes  handling  of  biases,  which  can  limit  accuracy,  ever  more 
important. 

5.3  Phased  Array  Radars 

5.3.1  Intrinsic  Observables 

Phased  array  radars  typically  measure  Range,  U,  and  V  direction  cosines  (hereafter  abbreviated  RUV)  [179]. 
These  are  usually  computed  for  every  detection  in  the  signal  processing.  The  U/V  direction  cosines  are 
referenced  to  the  grid  of  array  transmit/receive  elements  within  the  array  aperture.  The  U  and  V  reference 
directions  he  in  the  plane  of  the  aperture  and  are  orthogonal.  Thus  a  detection  at  U=0,  V=0  is  directly  on  the 
line  normal  to  the  aperture,  which  is  referred  to  as  the  boresite.  Range-rate  is  not  an  actual  observable  for 
most  systems,  but  some  of  the  newer  radar  systems  include  Doppler  waveforms  that  permit  measurement  of 
velocity.  Some  of  the  phased  array  tracking  sites  have  multiple  apertures  (sometimes  referred  to  as  faces). 
An  example  of  such  is  the  Upgraded  Early  Warning  Radar  in  Fylingdales,  England.  In  this  case,  the  RUV 
data  is  referenced  to  the  actual  aperture  making  the  detection. 

Provided  with  a  suitably  accurate  cue,  phased  array  radars  can  directly  track  an  object  based  on  the  pre¬ 
dicted  ephemeris.  If  the  object  is  uncued,  for  example  a  UCT,  the  track  is  maintained  by  using  a  sequential 
track  filter  to  predict  the  location  of  the  target  at  the  next  pulse  transmit  time.  Tracks  are  usually  updated 
every  few  seconds  to  keep  the  track  filter  estimates  accurate. 

Within  the  SSN,  the  data  is  typically  corrected  at  the  detection  level  for  system  biases,  tropospheric  and 
ionospheric  refraction  before  being  used  in  the  track  filters.  Since  the  RUV  data  is  registered  to  the  aperture 
boresite,  array  phase  center  and  aperture  rotation  around  the  boresite  vector,  the  data  is  tied  to  the  terrestrial 
reference  frame. 

Biases 

Phased  array  radar  biases  arc  caused  by  three  major  classes  of  problems:  environmental,  electronic,  and 
pointing.  These  biases  have  different  signatures  in  the  intrinsic  observable  measurement  space  and  also 
range,  azimuth,  elevation  (RAE).  The  environmental  errors  are  typically  dependent  on  RAE,  and  do  not 
map  1 : 1  into  the  RUV  coordinates  of  the  radar.  If  the  boresite  is  pointed  at  the  horizon,  elevation  will  align 
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with  one  of  the  direction  cosine  reference  axes  in  the  middle  of  the  FOV,  but  will  cross  the  axes  further  off 
boresite.  The  electronic  errors  are  strong  functions  of  RUV.  Operationally,  pointing  errors  tend  to  small  and 
can  be  aligned  with  RAE  or  RUV  coordinates.  Therefore  calibration  of  radar  data  biases  needs  to  account 
for  corrections  in  at  least  RAE  and  RUV,  and  in  some  cases  need  to  account  for  the  bore  site  orientation. 
Biases  in  the  intrinsic  observables  are  usually  corrected  at  the  detection  level  before  being  used  in  the  sensor 
level  processing. 

Troposphere  and  Ionosphere  Refraction  Errors  For  phased  array  apertures  oriented  with  the  boresite 
pointed  near  the  horizon,  a  primary  source  of  error  is  refraction.  Corrections  are  typically  made  for  the 
troposphere;  however,  the  model  accuracy  is  limited  to  approximately  10%  [26,  27,  53,  203].  Residual 
errors  can  result  in  a  measurement  accuracy  floor  at  low  elevations. 

Tropospheric  refraction  models  represent  a  long  term  mean  for  the  wet  and  dry  components  of  the 
atmosphere.  Most  models  assume  that  the  collection  is  uniform  in  azimuth.  Local  weather,  such  as  fronts 
and  storms,  along  the  line  of  sight  can  vary  significantly  from  the  model  and  induce  errors  in  both  elevation 
and  azimuth  [26].  Figures  5.3.1  and  5.3.2  show  typical  tropospheric  refraction  errors  in  the  elevation  and 
range  directions  before  and  after  correction.  Below  approximately  20°  elevation  the  tropospheric  refraction 
errors  are  significant.  The  accuracy  of  the  correction  depends  on  the  model,  and  the  averaging  period  for  the 
local  weather  data  used  in  the  correction. 

Figure  5.3.1  Elevation  angle  error  due  to  tropospheric  refraction.  Vehicle  height  is  200  km.  Maximum 
remaining  error  after  correction  is  based  upon  exponential  troposphere  ray-tracing.  Results  are  applicable  to 
most  tracking  sites  and,  in  particular,  those  near  Miami,  FL;  Flagstaff,  AZ;  Brownsville,  TX;  Washington, 
DC;  Fresno,  CA;  Bismarck,  ND;  and  Flamilton,  Bermuda.  Source:  Schmid  [234], 


Ionospheric  refraction  affects  longer  wavelengths  more  strongly.  Figures  5.3.3  and  5.3.4  show  typical 
ionospheric  refraction  errors.  The  effects  are  significant  at  VHF/UHF,  less  so  at  L  band,  and  relatively  minor 
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Figure  5.3.2  Tropospheric  range  error  remaining  after  correction  versus  elevation  angle.  The  error  remain¬ 
ing  after  correction  represents  the  difference  between  ray-traces  at  Ns  =  377  and  Ns  =  313  (standard 
atmosphere).  Source:  Schmid  [234]. 


at  X  band.  There  is  a  time  of  day  dependence,  and  a  strong  dependence  on  site  latitude,  with  stronger  effects 
seen  at  equatorial  sites.  (In  both  Figures  5.3.3  and  5.3.4,  the  ordinate  “Me”  stands  for  mega  cycles,  i.e., 
megahertz  “MHz.”) 

Tropospheric  and  ionospheric  errors  can  vary  over  a  long  track.  If  the  object  is  tracked  horizon  to 
horizon,  the  errors  will  have  the  same  sign  at  the  beginning  of  the  track  and  the  end  of  the  track.  Very 
strong  local  weather,  such  as  large  thunderstorms,  can  result  in  significant  changes  in  the  tropospheric  errors 
through  a  track.  A  scale  correction  to  the  model  can  often  correct  the  errors  for  a  given  track  or  time  period. 

Electronic  Biases.  Electronic  biases  are  the  result  of  errors  in  the  knowledge  of  the  electronic  performance 
parameters  in  the  radar.  In  some  cases  these  are  errors  in  the  knowledge  of  the  parameters,  and  in  some  cases 
limits  in  the  stability  of  the  parameters.  There  are  simple  constant  biases,  for  example  a  constant  phase  error 
in  a  phase-phase  steered  system  that  results  in  a  U  or  V  bias.  There  arc  also  scan-dependent  biases  that  can 
result  from  thermal  effects  on  the  array,  causing  an  error  in  the  knowledge  of  the  element  spacing,  as  well 
as  phase  dependent  errors.  Thermal  effects  cause  scale  errors  of  the  form  Ejj  =  xU  and  Ey  =  xV,  where 
Ejj  and  Ey  are  the  errors  in  the  U  and  V  components  respectively.  If  the  array  is  large  enough  there  could 
be  temperature  gradients  across  the  array,  complicating  the  form  of  the  bias.  For  arrays  pointed  at  or  near 
the  horizon,  residual  tropospheric  effects  on  a  given  short  track  (not  passing  point  of  closest  approach)  can 
generate  errors  that  look  like  scale  errors  if  constant  biases  are  also  computed.  These  biases  can  vary  from 
track  to  track  depending  on  the  design  of  the  system.  For  IOD  purposes  they  look  like  a  pure  bias,  however 
for  multipass  orbit  determination  using  multiple  days  of  data,  some  of  these  biases  look  like  a  long  period 
structured  noise. 
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Figure  5.3.3  Ionospheric  maximum  elevation  angle  error  for  5°  elevation.  Error  decreases  as  elevation  angle 
increases  above  5°.  Source:  Schmid  [234]. 


Maximum  elevation  error  (mr) 


Figure  5.3.4  Ionospheric  maximum  range  error  for  5°  elevation.  Error  decreases  as  elevation  angle  increases 
above  5°.  Source:  Schmid  [234]. 


Maximum  range  error  (meters) 


Pointing  Biases.  Pointing  biases  are  the  error  in  the  knowledge  of  the  array  pointing  and  affect  the  U  and 
V  direction  cosines.  Usually  these  are  computed  based  on  survey  results.  With  newer  more  accurate  tracking 
systems  coming  online,  these  errors  may  be  observable.  The  correction  requires  knowing  the  direction  of 
the  boresight,  which  can  be  expressed  in  azimuth  and  elevation,  as  well  as  the  rotation  about  the  boresight. 
The  direction  of  the  boresight  may  be  absorbed  by  other  biases  estimates  in  calibration,  but  the  rotation 
about  the  boresight  will  not. 

Noise 

U  and  V  direction  cosine  noise  on  a  radar  is  driven  by  the  wavelength,  beam  width  and  waveform  processing. 
In  general  the  noise  will  decrease  as  the  aperture  size  increases.  Beam  broadening  with  the  angle  off  of 
boresight  will  increase  the  noise  by  a  factor  of  sec(0),  where  6  is  the  angle  off  boresight.  Range  noise  is  a 
function  of  the  resolution  set  by  the  waveform  used  for  each  detection.  At  60°  off  boresite,  the  phased  array 
angle  data  is  twice  as  noisy  as  it  is  at  boresite. 
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Figure  5.3.5  Nominal  noise  profile  as  a  function  of  detection  SNR.  This  curve  assumes  a  nominal  20: 1  BSR 
at  infinite  SNR.  This  curve  is  generally  descriptive  for  angle  and  range.  Noise  values  are  normalized  against 
the  noise  floor,  hence  unitless,  and  do  not  represent  any  actual  system. 


SNR(dB) 


RUV  noise  variation  is  also  driven  by  signal  to  noise  ratio  (SNR)  dependent  noise.  In  general  noise 
decreases  as  SNR  increases,  asymptotically  approaching  a  noise  floor  [Figure  5.3.5].  The  noise  floor  itself 
can  vary  sensor  by  sensor,  even  in  a  family  of  sensors.  The  ratio  of  this  noise  floor  to  the  beam  width  is 
referenced  to  as  the  beam  split  ratio  (BSR)  at  infinite  SNR.  These  values  can  differ  for  range,  U  and  V. 
Typical  BSR  values  are  10  or  20:1. 

In  general,  the  detection  data  noise  varies  over  a  track.  SNR  and  the  angle  off  boresight  are  not  constant 
along  a  track.  Likewise,  waveforms  typically  vary  over  a  track.  The  details  of  this  variation  are  driven  by 
the  sensor  track  design  and  the  way  the  system  accounts  for  radar  cross-section  (RCS)  fluctuation.  Some 
systems  put  more  power  (i.e.  use  a  higher  SNR)  at  the  start  of  the  track  to  settle  the  track  filter.  Later  in 
the  track  a  lower  SNR  or  update  rate  is  used  to  preserve  system  capacity.  If  the  SNR  is  reported  for  each 
detection,  the  noise  floor  is  known,  and  the  correct  RCS  is  used  (peak  vs  average),  then  the  noise  variance 
for  a  given  detection  can  be  accurately  computed. 

Provided  that  sensor  has  sufficient  capacity,  the  quality  of  the  data  will  not  depend  on  the  number  of 
objects  in  track.  However,  as  the  sensor  capacity  is  saturated,  for  example  in  a  breakup,  less  capacity 
may  be  allocated  for  each  track.  This  behavior  will  be  very  dependent  on  the  details  of  the  design  and 
implementation  of  each  sensor. 

5.3.2  Sensor  Level  Processing 

Phased  array  radars  are  capable  of  processing  thousands  of  detections  a  second.  In  the  past,  data  was 
compressed  into  the  metric  observations  required  by  the  JSpOC.  This  compression  of  the  tracking  data  to 
a  smaller  set  of  data  points  that  theoretically  cany  the  same  information  is  a  common  practice  in  orbit 
determination  environments.  This  is  done  for  spacecraft  mission  navigation,  and  high  precision  geodetic 
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applications.  However,  in  the  case  of  the  SSN,  the  metric  observations  reported  do  not  reflect  the  actual 
detection  data.  For  the  SSN,  the  phased  array  radar  detections  are  transformed  from  their  natural  coordinate 
system  (RUV)  to  the  range,  azimuth,  elevation  and  range-rate  information  reported  in  the  metric  observa¬ 
tions.  Range -rate  is  produced,  even  if  there  was  not  intrinsic  range  rate  observable. 

Metric  Observation  Processing 

Some  phased  arrays  lack  the  angle  resolution  in  a  single  detection  to  meet  metric  observation  accuracy 
requirements.  The  detection  data  is  filtered  to  reduce  the  noise,  and  compress  the  data.  In  legacy  systems 
the  metric  observation  data  is  computed  on  the  basis  of  the  track  filter  state.  Once  the  covariance  in  the  filter 
has  reached  the  required  level,  the  metric  observations  arc  computed  at  a  periodic  interval  as  necessary  to 
satisfy  the  system  tasking.  These  filtered  values  arc  currently  reported  in  the  B3  Metric  observation  message, 
however  the  uncertainty  data  is  not. 

A  phased  array  radar  track  filter  is  intended  to  keep  the  comparatively  wide  radar  beams  on  target.  A 
simplified  model  is  used  to  propagate  the  state  vector  and  covariance  in  the  centered  rotating  frame,  and  state 
updates  arc  done  in  the  radar  RUV  coordinates.  Initialization  of  the  track  filter  state  depends  on  whether  the 
target  was  cued.  If  cued,  the  cue  can  be  used  to  start  the  filter.  If  uncued,  the  initialization  parameters  arc 
computed  from  the  initial  detections,  and  the  rate  terms  by  differing  the  values.  While  this  track  filtering 
process  works  well  for  pointing  the  beams,  it  can  potentially  introduce  errors  into  the  metric  observations. 

Depending  on  the  track  filter  tuning,  the  errors  in  the  initial  state  can  be  carried  through  the  entire  track. 
In  some  cases  the  bias  errors  in  the  initial  points  could  be  noise,  or  tropsospheric/ionospheric  errors,  or 
differences  in  processing  of  detect,  verify,  initial  track  maintenance,  or  track  waveforms.  Use  of  the  track 
filter  state  to  compute  the  metric  observations  produces  observations  that  have  reduced  noise,  however,  the 
observations  arc  correlated  to  each  other  across  the  track,  and  arc  not  of  uniform  weighting.  Further,  the  B3 
metric  observation  format  lacks  any  mechanism  to  report  the  observation  uncertainties  from  the  sensor. 

Accurate  range  data  computation  requires  computation  of  a  Range-Doppler  coupling  constant  for  cor¬ 
rection  of  the  range  data.  This  requires  an  accurate  estimate  of  the  range-rate.  Track  filter  errors  in  range-rate 
will  cross  over  into  the  range.  This  can  lead  to  an  error  in  range  that  is  variable  across  the  track.  Ideally  this 
correction  needs  to  be  recomputed  once  the  track  is  complete. 

Use  of  the  metric  observations  to  produce  a  calibrated  covariance  will  be  subject  to  a  number  of  limi¬ 
tations:  1)  the  distribution  of  the  observations  over  a  track  does  not  match  the  detection  data,  2)  the  metric 
observation  Azimuth  and  Elevation  does  not  map  directly  into  the  intrinsic  U/V  direction  cosine  space  used 
by  the  radar,  3)  unless  the  radar  includes  a  Doppler  waveform,  the  range  rate  information  is  not  a  real 
measurement;  using  it  as  such  will  bias  the  covariance. 

Sensor  Level  Calibration 

Calibration  of  the  large  phased  array  radars  is  done  at  multiple  levels.  The  phase  delays  and  related  biases 
in  the  array  arc  calibrated  using  a  mix  of  Fourier  gages  (known  signal  feeds)  as  well  as  subarray  calibration 
on  satellites,  and  metric  data  calibration.  Correction  factors  are  applied  at  many  levels  within  the  system,  as 
dictated  by  the  hardware  design.  These  calibrations  arc  done  infrequently,  generally  on  an  as  needed  basis. 
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Of  primary  importance  for  this  report  is  the  metric  calibration. 

Most  of  the  radars  calibrate  against  targets  with  known  ephemerides  to  determine  Range,  U,  V  correc¬ 
tions.  Generally  these  arc  treated  as  constant  biases  across  the  FOV,  but  it  is  possible  to  compute  higher 
order  corrections.  This  is  limited  in  practice  by  correlation  between  environmental  phenomena,  such  as  tro¬ 
pospheric  refraction  modeling  errors,  and  higher  order  terms,  such  as  U  or  V  scale  biases  for  sensors  pointed 
near  the  horizon. 

The  current  operational  practices  favor  using  high  SNR  observations  for  calibration.  For  some  of  the 
existing  systems  this  is  possible  only  near  boresight  (due  to  beam  steering  losses).  The  result  is  that  the 
sensor  is  not  necessarily  calibrated  over  the  full  FOV;  instead,  the  results  may  be  dominated  by  bias  features 
specific  to  the  area  near  boresight. 

One  aspect  of  the  Russian  Space  Surveillance  System  that  is  different  from  the  US  is  the  radar  tracking 
of  objects  in  LEO  and  the  reporting  of  observations.  Rather  than  report  a  set  of  observations  as  we  do  they 
report  the  object  state  at  one  time  and  the  diagonal  paid  of  the  covariance.  The  state  reported  is  the  range, 
azimuth  and  elevation  and  the  three  rates.  Azimuth  is  measured  in  the  radar  frame,  not  from  North  as  is 
usually  done.  In  this  frame  the  covariance  is  mostly  diagonal  so  only  the  diagonal  is  transmitted  and  this 
reduces  the  amount  of  date  transmitted  for  each  object  tracked. 

5.3.3  Initial  Orbit  Determination 

All  the  phased  arrays  generate  a  state  vector  in  the  track  filter  for  all  cases  where  a  track  is  successful. 
This  estimate  includes  the  covariance,  which  represents  the  covariance  of  a  Kalman  filter  tuned  for  track 
maintenance.  If  the  sensor  is  a  multi-mission  sensor  (e.g.  providing  missile  warning  and  space  surveillance 
support)  the  IOD  state  vector  and  covariance  are  reported  to  some  customers.  However,  the  track  state  vector 
and  covariance  data  is  not  reported  to  the  JSpOC.  Sometimes  the  elsets  are  sent  to  the  JSpOC,  but  the  current 
format  does  not  include  the  covariance. 

It  is  possible  for  the  JSpOC  to  perform  IOD  from  the  metric  observations.  The  track  filter  based  com¬ 
putation  of  metric  observations  represents  a  “lossy”  data  compression  technique,  like  JPEG.  It  is  “lossy” 
because  the  effective  track  arc  length  is  reduced.  This  is  particularly'  true  of  the  staid  of  the  track  where  the 
filter  is  converging.  The  result  is  that  IOD  based  on  the  metric  observations  can  have  degraded  performance, 
especially  for  short  UCT  tracks. 

5.4  Other  Radar  Types 

5.4.1  Mechanical  Radars 

The  primary  difference  between  dish  radars  and  phased  arrays  for  purposes  of  catalog  maintenance  is  the 
effect  of  the  mount  type.  The  mechanical  mount  will  drive  the  biases  in  the  pointing  due  to  biases  in 
measurement  of  the  orientation,  sag,  misalignment  etc.  If  the  mount  is  an  az-el  type  mount,  estimating  the 
corrections  in  that  frame  is  best.  Noise  for  the  angle  data  will  still  be  driven  by  the  aperture  size  and  SNR. 
Beam  broadening  will  not  be  a  factor  in  the  noise.  There  can  be  a  small  component  of  noise  added  by  the 
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transducers  that  measure  the  mount  location. 

5.4.2  Hybrid  Radars 

Hybrid  radars,  having  a  phased  array  mounted  on  a  mechanical  mount  share  characteristics  of  both  me¬ 
chanical  and  phased  array  radars.  The  mechanical  mount  will  add  biases  in  the  pointing  due  to  biases  in 
measurement  of  the  orientation,  sag,  misalignment  etc.  This  will  apply  on  top  of  the  phased  array  biases  in 
the  RUVD  coordinate  system.  If  the  mount  is  an  az-el  type  mount,  it  may  be  necessary  to  estimate  correc¬ 
tions  in  the  RAE  and  RUV  frames  simultaneously.  Noise  for  the  angle  data  will  be  driven  by  the  aperture 
size,  SNR,  and  beam  broadening.  There  can  be  a  small  component  of  noise  added  by  the  transducers  that 
measure  the  mount  location. 

5.5  Optical  Tracking 

Optical  tracking  data  provides  angle  accuracies  that  arc  much  better  than  radar  data,  with  correspondingly 
low  noise.  The  lack  of  range  data  complicates  IOD.  Being  referenced  to  the  celestial  frame,  and  expressed 
in  Right  Ascension  and  Declination,  there  is  no  need  to  account  for  refraction  corrections  and  biases,  mount 
issues,  etc.,  like  phased  array  radars.  Some  future  optical  sensors  arc  using  RAE  mounts  and  reporting  in  the 
RAE  frame.  This  will  be  subject  to  differences  between  Earth  Center  Fixed  and  Celestial  reference  frames, 
just  like  the  phased  array  radars. 

Optical  tracking  systems  use  large  telescopes  equipped  with  charge-coupled  devices  (CCDs)  to  detect 
photons  from  the  target.  The  mount  types  and  orientation  vary  based  on  the  telescope  and  platform.  Since 
CCDs  work  by  integrating  the  incident  light  energy  striking  a  given  cell,  the  motion  of  the  source  over 
the  CCD  array  can  strongly  affect  the  sensitivity  and  quality  of  the  track.  Most  systems  operate  in  one  of 
two  basic  tracking  modes:  scan  while  track,  deep  stare,  or  both  depending  on  the  tasking.  If  the  object  is 
tumbling  during  the  track,  the  signal  can  vary  over  the  track  as  well.  Newer  processing  techniques  for  track 
before  detect  arc  being  researched,  and  could  be  added  to  future  systems. 

5.5.1  Intrinsic  Observables 

The  image  scanned  from  the  CCD  forms  the  basic  set  of  observable  data.  Tracks  and  features  arc  scanned  in 
the  CCD  image,  and  compared  to  a  stellar  background.  This  registers  the  image  to  the  celestial  background 
and  celestial  reference  frame.  After  registration  to  the  stellar  catalog,  the  tracks  can  be  identified  (the 
specific  star  catalog  varies  from  site  to  site).  The  photometric  flux  of  the  object  can  be  computed  based  on 
the  intensity  of  the  signal  at  the  CCD.  This  flux  value  can  then  used  to  compute  a  visual  magnitude. 

In  a  deep  stare  mode,  the  sensor  is  pointed  at  a  fixed  point  in  the  inertial  sky.  Returns  from  space  objects 
form  tracks  on  the  CCD,  while  stars  show  as  dots.  These  tracks  arc  scanned  for  in  the  image  and  identified. 
In  scan  while  track  mode,  the  telescope  is  programmed  to  follow  a  specified  track  that  is  intended  to  match 
the  motion  of  the  target.  The  stellar  background  may  appeal-  as  streaks,  while  the  intended  (or  hoped  for) 
target  will  be  a  bright  dot.  Velocity  errors  in  the  anticipated  track  will  result  in  image  smearing.  This  method 
allows  for  longer  integration  time  on  the  track,  allowing  fainter  objects  to  be  tracked. 
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Biases 

Biases  can  arise  from  multiple  sources:  stars  in  field  of  view,  sensor  resolution  (telescope  size),  CCD  quality, 
and  track  smearing. 

Noise 

Noise  is  driven  by:  target  brightness,  Sensor  Resolution  (telescope  size),  CCD  quality,  etc.  Track  smearing 
and  atmospheric  blurring  can  also  affect  noise. 

5.5.2  Sensor  Level  Processing 
Metric  Observation  Processing 

Optical  angle  measurements  arc  registered  to  stellar  background.  Removal  of  the  background  signal  allows 
tracks  to  be  identified.  The  track  data  is  filtered  to  determine  the  track  end  points  and  observation  rates. 
These  form  the  basis  of  the  metric  observations. 

Biases  can  arise  from  the  presence  of  stars  on  or  near  the  track,  and  the  selection  of  stellar  catalog  and 
reference  stars  in  the  imaged  field.  A  star  in  near  a  track  can  result  in  a  bias  in  the  computed  track  end  point 
position.  The  signature  of  the  star  and  the  target  arc  close  enough  to  form  one  elongated  blob,  which  corrupts 
pixel  centering  algorithms.  A  weak  geometry  or  mismatch  in  the  selection  of  reference  stars  can  result  in  a 
poor  computation  of  the  location  of  the  track.  The  reference  catalogs  vary  between  the  systems  and  JSpOC 
processing,  potentially  introducing  biases  between  systems.  An  additional  source  of  bias  is  sensor  location 
errors. 

Noise  in  the  metric  observations  is  driven  by  the  data  noise,  as  well  as  the  accuracy  of  the  track  fit.  In 
addition,  the  B3  message  format  lacks  sufficient  significant  figures  for  the  measurements,  introducing  an 
accuracy  floor  in  the  observations  for  some  orbital  regimes. 

Sensor  Level  Calibration 

Internal  calibration  of  the  sensor  is  performed  to  calibrate  the  visual  magnitude  accuracy  as  well  as  the 
telescope  boresight  to  support  mission  operations.  Since  the  data  itself  is  registered  to  the  stellar  background, 
the  biases  applied  internal  to  the  sensor  for  pointing  are  not  a  factor  in  the  data  returned  to  the  JSpOC. 

5.5.3  IOD 

Angles-only  IOD  from  short  tracks  is  subject  to  large  orbit  uncertainties,  if  it  is  even  possible  due  to  the  short 
tracks  involved.  The  assumption  of  some  of  the  orbit  parameters,  such  as  semimajor  axis  or  eccentricity,  is 
required  depending  on  the  theory  being  used.  The  IOD  results  can  be  used  for  short  term  planning,  and  are 
not  generally  reported  to  the  JSpOC. 
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5.6  Current  JSpOC  Calibration 

JSpOC  currently  calibrates  the  sensors  used  in  the  catalog  on  a  periodic  basis.  An  independent  set  of 
high  precision  orbit  ephemeris  data  is  used  to  process  observations  of  a  limited  number  of  targets  to  assess 
the  standard  deviation  and  biases  in  the  data.  The  standard  deviation  of  and  biases  of  RAE  residuals  arc 
computed  for  radar  sites,  and  right  ascension/declination  residuals  for  optical  sties.  The  results  represent  an 
average  over  many  passes.  These  values  arc  then  applied  to  the  catalog  processing. 

This  approach  does  not  take  into  account  the  variability  in  formal  uncertainty  for  the  radar  sites  as  a 
function  of  angle  off  the  boresight.  The  elevation  noise  and  bias  will  capture  most  refraction  modeling 
errors.  However,  the  radar  noise  and  biases  do  not  map  linearly  into  azimuth.  Consequently  the  computed 
azimuth  noise  will  not  accurately  reflect  track  accuracy  throughout  the  field  of  view.  For  upward  looking 
sensors,  such  as  the  now  decommissioned  VHF  fence  and  possibly  the  future  S  band  fence,  the  singularity 
in  the  RAE  frame  that  occurs  directly  overhead  of  the  sensor,  coupled  with  non  uniform  data  distributions, 
can  lead  to  a  calibration  computation  that  does  not  reflect  the  actual  system  behavior. 

5.7  Summary 

Correct  modeling  and  assessment  of  sensor  noise  and  bias  performance  is  necessary  for  correct  covariance 
estimation.  The  corrections  and  models  that  arc  required  to  be  used  for  optimal  assessment  differ  remark¬ 
ably  between  sensor  types.  Radar-  systems  require  correction  for  refraction  effects,  as  well  as  other  system 
biases  in  order  to  accurately  determine  an  orbit  from  the  data.  The  optical  data  is  registered  to  the  stellar 
background,  eliminating  the  need  for  refraction  correction  models.  However,  optical  sensors  do  not  provide 
range.  Overall  the  radar  and  optical  sensors  are  complementary,  but  collect  modeling  is  required  to  ensure 
they  produce  orbits  in  the  same  effective  reference  frame. 

The  reported  metric  observations  for  phased  array  radars  do  not  match  the  intrinsic  measurements. 
Phased  array  radars  work  in  the  RUV  frame,  as  opposed  to  the  Time,  Elevation,  Range  and  Range-rate 
(TEARRR)  form  used  in  the  B3  data.  Most  of  the  radars  do  not  intrinsically  measure  range-rate.  Generation 
of  the  metric  observations  from  the  track  filter  can  be  an  added  source  of  error  in  the  data,  and  results  in 
correlation  between  the  observations  that  is  not  reflected  in  the  JSpOC  calibration. 

Phased  array  radar  noise  performance  is  strongly  determined  by  the  angle  off  boresight  and  the  track 
SNR.  The  distribution  of  the  detections  along  the  track  is  also  not  uniform.  This  results  in  a  variable  weight¬ 
ing  of  the  track  depending  on  location  in  the  FOV.  However,  the  current  B3  format  does  not  allow  reporting 
of  the  per  observation  uncertainties.  Radar  bias  performance  is  also  a  strong  function  of  the  RUV  frame.  By 
comparison  the  JSpOC  calibration  of  noise  and  biases  works  in  the  RAE  frame. 

In  general,  in  the  angular  components,  the  optical  data  is  much  more  accurate  than  the  radar  data.  The 
data  is  registered  to  a  given  celestial  frame  via  comparison  of  the  image  to  the  stellar  background.  However, 
different  star  catalogs  are  used  at  different  systems,  potentially  causing  biases  between  optical  sensors,  and 
the  optical  and  radar  data  sets.  As  with  the  radar  data,  observation  uncertainties  are  nor  reportable  in  the 
B3  metric  observation  format.  In  addition,  the  B3  format  lacks  the  precision  to  fully  exploit  the  accuracy 
potential  of  the  optical  data  in  all  regimes. 
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5.8  Recommendations 

Significant  improvement  in  measurement  covariances  from  the  SSN  sensors  can  be  achieved  by  making 
changes  at  the  sensors  as  well  as  at  the  JSpOC.  The  current  sensor  and  JSpOC  architecture  is  rigid.  Nonethe¬ 
less  there  is  still  the  possibility  of  some  short  and  mid-term  actions  to  improve  uncertainty  realism.  A  future 
JSpOC  Mission  System  improvement  will  provide  a  significant  step  forward  in  terms  of  opportunity  for 
improvement,  but  additional  improvements  to  the  network  will  still  be  required.  The  capability  to  fully  and 
accurately  exploit  the  sensor  data  for  uncertainty  realism  will  need  to  come  in  later  developments. 

5.8.1  Short-Term 

In  the  short  term,  it  may  be  possible  to  realize  some  operational  improvement  by  development  and  use 
of  an  empirical  error  model  for  weighting  of  the  sensor  data  at  the  JSpOC.  This  would  have  to  use  the 
existing  B3  Metric  observation  format  data.  Error  models  would  need  to  be  developed  for  classes  of  sensors 
and  applied  to  individual  sensors.  These  models  would  still  provide  range  and  bias  correction  estimates; 
however,  they  would  make  use  of  basic  sensor  uncertainty  characteristics  to  develop  more  accurate  bias  and 
noise  estimates.  This  should  be  coupled  with  corresponding  changes  in  the  orbit  determination  software. 
Without  that  change,  the  improved  estimates  may  simply  be  wasted. 

Example  phased  array  bias  corrections  should  include  the  following  terms  as  a  core: 

Boresight  Azimuth  Elevation  and  Clock  Angle  (rotation  around  the  boresight):  These  arc  needed  to 
properly  transform  the  measurements  from  the  Metric  Observation  RAE  to  an  intrinsic  observable  RUV 
frame.  Most  of  the  existing  arrays  arc  symmetrical,  so  the  U/V  designation  is  arbitrary,  but  should  be 
consistently  handled.  However,  not  all  arrays  for  radars  that  may  be  used  arc  symmetrical,  so  care  should 
be  taken  to  accurately  reflect  the  orientation. 

Simple  bias  by  intrinsic  observable  type:  Where  possible  when  using  detection  data,  this  should  reflect 
the  waveform  type  used,  as  jumps  can  occur  between  waveforms.  When  metric  observations  arc  used,  this 
is  unnecessary.  It  may  be  necessary  to  break  this  bias  up  over  time,  as  some  biases  fluctuate  at  intervals  of 
hours  to  weeks. 

Scale  bias  for  each  intrinsic  observable  type:  This  will  handle  array  spacing  and  thermal  correction 
errors  on  phased  arrays.  Scale  errors  can  affect  range,  and  Doppler  as  well.  As  with  the  simple  biases,  it  may 
be  necessary  to  break  these  parameters  over  time  periods  to  account  for  drift  in  some  of  the  driving  factors 
For  example,  thermal  expansion/contraction  in  the  array  will  cause  scale  errors  in  the  U/V  measurements. 

For  a  typical  radar,  this  will  require  that  approximately  nine  parameters  be  estimated.  Note  that  range- 
rate  biases  should  not  be  included  unless  the  radar  actually  measures  Doppler  shifts  via  a  pulse-Doppler 
waveform.  However,  it  may  be  useful  for  characterization  of  metric  observation  performance. 

The  error  model  development  would  start  with  calibration  residuals  for  a  few  radars  on  a  per  metric 
observation  bias  over  a  90-day  period.  The  initial  step  would  be  to  plot  the  errors  as  a  function  of  azimuth, 
elevation,  range,  off-boresight  angles,  local  time,  etc.  Statistical  analysis  would  be  performed  to  identify 
dominant  relationships  as  a  function  of  track  geometry.  The  statistical  analysis  should  be  backed  up  by 
covariance  analysis  to  fully  understand  the  empirical  results.  Covariance  analysis  can  be  used  to  map  force 
model  errors  and  observation  modeling  biases  to  the  orbit  determination  and  prediction  accuracy,  allowing 
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the  top  contributors  to  be  identified.  This  will  require  tools  capable  of  computing  the  contribution  of  the 
force  model  errors  and  data  biases  to  the  orbit  determination  covariance,  as  well  as  the  effect  of  the  error 
on  the  prediction.  Such  covariance  analysis  tools  exist  (e.g.,  NASA’s  ORAN  [1 10,  199]  or  ODEAS  [256]); 
alternatively  Monte  Carlo  analyses  can  be  performed. 

Knowledge  of  the  top  modeling  error  contributions  to  the  covariance  can  be  used  to  develop  a  model  for 
a  given  class  of  sensor  to  apply  in  testing.  The  modified  software  including  the  error  model  can  be  evaluated 
against  the  baseline  software  and  assessed  using  the  precision  ephemeris  data.  The  realism  of  estimated 
noise-only  error  at  the  end  of  the  OD  arc  and  at  a  nominal  prediction  time  can  be  assessed.  If  significant  and 
meaningful  improvements  are  achieved,  then  modification  of  the  operational  software  can  be  explored. 

5.8.2  Mid-Term 

As  sensors  arc  upgraded,  there  will  be  the  opportunity  to  expose  and  use  the  actual  detection  data  directly. 
A  study  should  be  performed  to  assess  the  practical  noise  characteristics  for  each  sensor,  using  the  actual 
sensor  detection  data.  The  results  of  this  study  can  be  used  to  guide  development  planning  and  prioritization. 
Sensors  for  which  the  detection  data  will  provide  the  most  improvement  in  accuracy  and  uncertainty  realism 
can  be  prioritized  for  upgrade  first. 

The  detection  data  will  need  to  be  collected  from  the  sensor  systems  as  it  is  not  reported  currently. 
This  goes  a  step  further  than  the  short  term  recommendation  by  looking  at  the  actual  intrinsic  measurement 
quality  and  characteristics,  along  with  the  reported  (or  computable)  noise  variances.  This  will  allow  the 
impact  of  the  sensor  level  processing  (metric  observation  via  the  track  filter),  the  detection  distribution 
through  a  track,  and  the  per  detection  noise  variations  to  be  assessed. 

A  mid-term  operational  enhancement  that  may  be  practical  is  to  have  the  sensors  report  the  uncertainties 
in  the  metric  observations  to  the  JSpOC  via  the  current  messaging  architecture.  This  can  be  accomplished 
by  reporting  the  uncertainties  in  the  same  B3  format,  but  with  a  flag  to  indicate  that  they  are  the  uncertainties 
in  the  observations.  This  will  require  relatively  little  modification  to  each  sensor,  but  will  take  time  for  all 
the  sensors  to  be  upgraded.  JMS  Increment  2  should  be  able  to  make  use  of  this  data  however. 

5.8.3  Long-Term 

Long-term  developments  of  JSpOC  processing,  net-centric  interfaces  and  sensor  backends  will  provide  the 
opportunity  for  enhanced  sensor  data  exploitation  and  uncertainty  realism.  Some  of  these  efforts  will  by 
necessity  take  years  to  put  in  place.  Future  JSpOC  Mission  System  updates  should  provide  an  opportunity 
for  improvement  in  the  modeling  and  application  of  sensor  noise  statistics  that  should  improve  uncertainty 
realism. 

The  following  recommendations  are  meant  to  provide  guidance  for  research  and  development,  prototyp¬ 
ing,  and  new  systems  and  upgrades  requirements  development: 

1.  Where  possible,  intrinsic  observables  from  sensors  should  be  used.  Track  filter  derived  metric  ob¬ 
servations  should  not  be  used.  This  will  allow  correct  calibration  correction  and  weighting  of  the 
data  in  the  catalog  orbit  solutions.  The  ability  to  model  and  calibrate  the  sensor  boresight  pointing  is 
recommended  for  best  results. 
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2.  Detection  data  should  be  made  available,  which  in  turn  enhances  the  flexibility  in  data  processing. 
There  will  be  a  1  Ox  increase  in  the  number  of  observations.  If  the  volume  of  data  from  the  modern 
radars  exceeds  catalog  processing  limits,  then  a  “normal  point”  process  can  be  used  to  compress  the 
data.  This  technique,  used  by  the  International  Laser  Ranging  Service  [222],  compresses  the  data  by 
using  a  remove-restore  algorithm  to  filter  high  rate  laser  observations  over  a  given  time  window  to 
filter  high  rate  laser  observation  over  a  given  time  window  (e.g.,  the  Herstmonceux  algorithm  [243] 
).  The  remove-restore  does  not  produce  residual  error  artifacts  like  a  track  filter  does,  and  has  been 
demonstrated  to  support  1  cm  orbit  precision. 

3.  Sensor  noise  should  be  supplied  for  each  observation.  This  will  permit  modeling  of  the  degraded 
Phased  Array  performance  off  boresight  and  improve  radar  calibration  and  improve  the  accuracy  of 
the  orbit  solution  covariances.  In  the  full  implementation,  the  radar  should  report  the  observation 
variance  (noise)  for  each  point.  In  the  interim,  it  is  possible  to  develop  models  for  the  variation  in 
noise  across  the  radar  FOV,  however  this  will  not  account  for  SNR  variation. 

4.  Best  practices  from  the  wider  astrodynamics  community  for  reporting  of  detection  data  and  correc¬ 
tions  are  highly  recommended.  The  precision  radar  altimetry  community  has  successfully  employed 
a  process  for  over  20  years  where  the  raw  measurement  and  all  corrections  are  reported  in  each  data 
record.  This  allows  for  consistent  processing  of  data  from  multiple  sensors,  leading  to  more  accurate 
products,  and  error  estimates.  Examples  of  information  that  should  be  included  for  SSN  data  include: 

•  Raw  measurements 

•  Local  Weather  (Pressure,  Temperature,  Humidity,  Total  Electron  Count,  etc) 

•  Troposphere  refraction  corrections 

•  Ionosphere  refraction  corrections 

•  Range-Doppler  coupling  factor,  correction,  aberration,  and  Doppler  source 

•  Other  applied  bias  corrections  and  bias  correction  uncertainties,  by  correction 

•  Reference  star  catalog  or  Reference  boresight  direction  (as  appropriate  for  the  sensor) 

•  Observation  variances  (noise) 

•  Observation  SNR  for  radar 

These  corrections  should  be  reported  for  the  actual  detection  data,  as  well  as  any  compressed  data 
reported  to  the  JSpOC. 

5.  Calibration  of  sensor  data  should  be  performed  in  the  natural  coordinate  system  of  the  reported  intrin¬ 
sic  observables.  This  will  improve  the  accuracy  of  the  orbit  product  covariance. 

6.  IOD  state  vector  products  should  be  routinely  provided.  This  should  include  the  covariance  as  well  to 
support  correlation  and  planning  product  generation.  While  this  data  may  not  use  these  in  routine  pro¬ 
cessing,  these  products  are  very  useful  for  first  tracks  of  UCTs  and  new  launches/releases.  Publishing 
the  data  will  enable  more  flexible  mission  operations  under  JSpOC  policy  management.  It  should  be 
noted  that  the  sensor  provided  covariance  accuracy  may  be  of  limited  by  the  sensor  processing. 
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7.  Availability  of  detection  data  through  a  track,  as  well  as  the  uncertainties,  will  allow  the  JSpOC  to 
tailor  their  processing  of  the  tracks  as  needed.  This  may  render  many  of  the  existing  tasking  codes 
obsolete,  and  simplify  tasking.  This  aspect  should  be  investigated  and  reviewed. 

8.  The  Star  Catalog  used  to  process  the  optical  tracking  data  needs  to  be  coordinated  across  the  SSN. 
This  catalog  should  be  kept  up  to  date,  so  the  sensor  data  can  be  registered  to  commonly  used  refer¬ 
ence  frames  in  the  wider  astrodynamics  community.  At  this  point  a  Hipparcos  mission  derived  Star 
Catalog,  or  later,  should  be  considered.  Either  the  star  catalog  needs  to  be  standardized  for  all  re¬ 
ported  observations  from  all  sensors,  or  the  catalog  used  at  the  sensor  needs  to  be  indicated  in  the  data 
records. 

9.  The  latest  International  Astronomical  Union  reference  frames,  including  the  Celestial  Intermediate 
Origin  (CIO)  based  system  used  for  the  Hipparcos  Star  Catalog  data,  should  be  used.  This  will 
improve  co-registration  of  the  optical  and  radar  data,  as  well  as  make  the  resulting  space  catalog 
products  more  interoperable  with  the  wider  astrodynamics  community.  Future  enhancements  arc 
expected  to  provide  a  significant  improvement  in  this  area. 

10.  Another  alternative  that  would  reduce  the  amount  of  data  transmitted  would  be  to  use  the  Russian 
approach,  but  report  the  complete  track  covariance. 
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Chapter  6 


The  Inverse  Problem:  A  Brief  Review  of 
Estimation  Formulations  Used  in  Space 
Surveillance 


Having  presented  the  uncertainties  in  sensor  level  processing,  dynamical  models  including  atmospheric 
models,  this  chapter  surveys  some  of  the  formulations  and  methods  for  the  inverse  problem  of  extracting 
a  state,  e.g.,  position  and  velocity,  from  the  sensor  reports.  The  general  subject  of  linear  and  nonlinear 
estimation  is  presented  in  the  extensive  literature  of  articles,  texts,  and  reference  books,  some  classical  and 
some  of  more  recent  vintage.  In  particular',  many  of  the  problem  formulations  and  methods  in  this  chapter 
can  be  found  in  the  books  by  Escobal  [86],  Jazwinski  [139],  Anderson  and  Moore  [8],  Gelb  [96],  Bar- 
Shalom,  Li,  and  Kirubarajan  [25],  Crassidis  and  Junkins  [55],  and  Sarkka  [233].  A  more  recent  handbook  on 
nonlinear  filtering  is  that  of  Crisan  and  Rozovskii  [56].  Thus,  the  objective  in  this  chapter  is  to  briefly  survey 
some  of  the  formulations  and  methods  for  nonlinear  estimation  including  filtering,  smoothing,  prediction, 
and  batch  estimation  that  are  currently  used  in  the  astrodynamics  community.  In  particular,  a  goal  is  to 
estimate  the  state  and  its  uncertainty  at  the  time  of  a  sensor  report.  The  next  chapter  surveys  some  of  the 
methods  used  in  predicting  this  uncertainty  forward  in  time. 

6.1  The  Measurement  and  Dynamic  Equations 

The  problem  of  estimating  the  dynamic  state  of  an  object  assumes  a  stochastic  differential  equation  and 
a  sequence  of  measurements  Zk  =  {z\. . . . ,  zk }  at  the  respective  times  {fi, . . . ,  tk }.  The  measurement 
equation  is  normally  written  as 

zk  =  hk(x(tk))  +  uk  (k  >  1)  (6.1.1) 

wherein  the  noise  sequence  is  generally  modeled  as  independent  zero  mean  Gaussian  random  vari¬ 

ables  with  covariance  R?.  (More  generally,  the  noise  sequence  can  be  modeled  by  a  the  corresponding 
sequence  of  independent  probability  density  functions,  say  {pVi  (izj)}k=1  )■  hk(x)  represents  the  transfor¬ 
mation  of  the  state  x  of  an  object  to  the  space  of  the  measurement  z  at  time  tk.  (Parameters  that  may  be 
present  in  the  measurement  equation  have  been  suppressed.) 
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6.1.  THE  MEASUREMENT  AND  DYNAMIC  EQUATIONS  USED  IN  SPACE  SURVEILLANCE 

The  astrodynamic  equations  of  motion  define  a  second  order  system  of  equations  in  position  which  can 
then  be  converted  to  a  dynamic  state  x(t)  of  position  and  velocity  at  time  t.  The  corresponding  equations 
with  process  noise  or  stochastic  acceleration  can  be  formulated  as  a  continuous-time  model  of  first-order 
stochastic  differential  equation  of  the  form 

=  f(x(t),a,t )  +  G  (x(t),t)n(t),  (6.1.2) 

where  /  :  P  x  Rp  x  1  ->  Mn,  G  :  Mn  x  M  — >  Mnxm,  a  e  Mp  is  a  vector  of  the  system  parameters, 
and  n(t)  is  an  m-dimensional  representation  of  the  stochastic  acceleration  or  process  noise.  The  function 
f  encodes  the  deterministic  force  components  of  the  dynamics  (e.g.,  gravity,  drag,  solar  radiation  pressure, 
etc.)  and  parameters  a.  The  term  G(x(t),  t)  n(t)  should  encode  the  uncertainty  in  the  dynamic  models  and 
space  environment.  Several  examples  of  stochastic  acceleration  are  presented  in  the  Section  6.1.1.  In  each 
case  the  dynamics  and  process  noises  are  reduced  to  the  form  in  equation  6.1.2  in  which  n{t )  is  a  white 
zero  mean  Gaussian  process.  In  addition,  cc(to),  n(t),  and  {zxfc}  are  assumed  to  be  independent. 

While  the  astrodynamic  equations  of  motion  are  formulated  naturally  as  a  stochastic  differential  equa¬ 
tion  (6.1.2)  with  the  discrete  measurement  or  sequence  report  sequence  (6.1.1),  many  of  the  methods  of 
estimation  are  developed  for  the  discrete  problem 


Xk  =  fk—i (®fc— i j  Wfc-t)  {k  >  1)  (6.1.3) 

zk  =  hk(xk,uk)  (k  >  1)  (6.1.4) 

where  xk  =  x(tk),  fk  '■  K"  xFx  Mm  — >  M",  {wk}  and  { vk }  are  independent  noise  sequences  that  are  also 
independent  of  xq.  Often  the  further  specializations  are  made  to  their  being  zero  mean  and  E[wk  w?  }  = 
Q k  Skj  and  E [uk  }  =  R/,  5kj.  In  addition,  xo  ~  W(*0)  Po)-  Both  fk  and  hk  may  additionally  contain 
parameters.  There  are  at  least  four  different  cases  in  which  one  can  use  the  discrete  formulation  (6.1.3)  as  a 
substitute  or  approximation  for  the  continuous  version  (6.1.2). 

1.  The  continuous  equations  of  motion  are  linear  and  the  process  noise  term  G(x(t),t)  n(t )  does  not 
depend  on  the  state  x. 

2.  The  stochastic  acceleration  term  G (x(t),t)  n(t)  =  0,  but  the  dynamics  are  otherwise  nonlinear. 

3.  The  continuous  stochastic  differential  equation  is  approximated  by  the  discrete  version  in  the  case  of 
small  process  noise. 

4.  The  stochastic  differential  equation  is  approximated  by  a  numerical  scheme  [163]. 

6.1.1  Examples  of  Process  Noise  for  Continuous  Dynamics 

Process  noise  or  stochastic  acceleration  is  often  used  by  the  astrodynamics  community  to  mitigate  modeling 
errors  as  well  as  sensor  biases.  Four  different  examples  are  provided  below  for  the  scalar  case;  however, 
there  are  analogous  developments  for  the  multidimensional  case.  Each  can  be  included  in  the  dynamics  and 
converted  to  a  stochastic  differential  equation  of  the  form  (6.1.2). 
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White  Noise 

The  basic  noise  process  n(t)  in  the  stochastic  differential  equation  (SDE)  (6.1.2)  is  zero-mean  white  noise, 

n(t)  =  m(t),  E[n(t)]  =  E[m(f)]  =  0 
defined  by  the  autocovariance  function 

s )  =  E[(m(t)  -  E[w(f)])  (m(s)  -  E[m(s)])T]  =  Q (t)  6(t  -  s ) 

The  choice  of  the  covariance  matrix  Q(t)  is  part  of  the  design  problem.  Bar-Shalom,  Li,  and  Kirubarajan 
[25]  discuss  this  design  for  a  nearly  constant  velocity  model  based  on  the  need  to  accommodate  accelerations 
of  a  pre-specified  strength. 

The  Ornstein  Uhlenbeck  Process 

The  Ornstein  Uhlenbeck  process  is  one  specified  by  the  stochastic  differential  equation 

dn(t)  =  p{p  —  n(t))  dt  +  a  dW ( t ) 

where  p  >  0,  p  and  a  >  0  arc  parameters  and  W (t)  denotes  the  Wiener  process.  In  physical  sciences,  the 
Ornstein-Uhlenbeck  process  is  rewritten  as  the  Langevin  equation 

=  p{p  -  n{t ))  +  aw(t), 

where  w(t)  is  zero  mean  white  Gaussian  noise  with  E\iu(t)'w(s)]  =  5(t  —  s).  Furthermore,  one  can  make  the 
long  term  mean  p  stochastic  to  another  stochastic  differential  equation  (SDE)  in  what  is  sometimes  called  a 
cointelation  SDE.  In  finance,  the  Ornstein  Uhlenbeck  process  is  called  the  Vasicek  model  for  governing  the 
evolution  of  interest  rates. 

One  can  solve  for  n(t)  to  obtain 

n(t)  =  exp  (—p(t  -  to))n(to)  +  p(l  —  exp  {—p{t  —  to)))  +  /  exp  {—p(t  -  s))adW (s) 

Jt0 

The  typical  parameters  p,  p,  and  a,  together  with  the  initial  condition  n(0),  completely  characterize  the 
dynamics,  which  can  be  characterized  as  follows. 

i.  p  represents  the  “long  term  mean.”  All  future  trajectories  of  n  will  evolve  around  a  mean  level  p  in  the 

long  term. 

ii.  p  >  0  denotes  the  “speed  of  reversion,”  i.e.,  it  characterizes  the  rate  at  which  trajectories  will  approach 

or  decay  onto  p  in  time; 

iii.  a  is  the  “instantaneous  volatility”  and  measures  the  amplitude  of  randomness  entering  the  system. 

Higher  a  implies  more  randomness. 
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iv.  The  quantity  a2/2 p  denotes  the  long  term  variance.  All  future  trajectories  of  r(t)  will  regroup  around 
the  long  term  mean  with  this  variance  after  a  long  time. 

Note  that  p  and  a  tend  to  oppose  each  other:  increasing  er  increases  the  amount  of  randomness  entering  the 
system,  but  at  the  same  time  increasing  p  amounts  to  increasing  the  speed  at  which  the  system  will  stabilize 
statistically  around  the  long  term  mean  p  with  a  corridor  of  variance  determined  also  by  p.  This  is  clear 
when  looking  at  the  long  term  variance,  a2 / 2 p  which  increases  with  a  but  decreases  with  p. 

One  can  compute  the  following  for  the  Ornstein  Uhlenbeck  process 

E [n(t)}  =  exp  ( -p(t  -  t0))n(t0)  +  p(l  -  exp  (-p(t  -  t0))) 

Cov[n(s),  n(f)]  =  E[(n(s)  —  E[n(s)])(n(t)  —  E[n(i)])] 

2 

=  —  exp(-p(s  +  t))(exp(2pmin(s,t)  -  1) 

2  p 
2 

=  — (exp(— pit  —  s))  —  exp(— pit  +  s)))  provided  s  <  t. 

2  p 

The  Ornstein  Uhlenbeck  process  fits  within  the  general  framework  above  by  state  augmentation.  To  see  this, 
consider  the  combined  system 


x'(t)  =  f(x(t),t)  +  G(x(t),t)  n(t), 
n'(t)  =  p(p,  —  n{t ))  +  a  w(t). 


Now  define  the  augmented  state  variable  y  =  [x,  n).  Then, 


y\t) 


x\t) 

f(x(t),t)  +  G(x(t),t)n(t) 

n'(t ) 

p{p  -  n(t)) 

f(y(t),t)  +  Gn(t). 


0 

crw(t) 


Therefore,  in  the  SDE  governing  the  augmented  state  y(t),  the  noise  process  is  white.  Finally,  we  note 
this  process  is  also  used  for  a  maneuvering  target  model  in  the  general  field  of  multiple  target  tracking  as 
described  in  the  survey  by  Li  and  Jilkov  [178]. 


First-Order  Gauss-Markov  Process 

The  first-order  Gaussian-Markov  process  (GMP1)  is  a  special  case  of  the  Ornstein  Uhlenbeck  process  (//  = 
0)  and  is  defined  by  the  stochastic  differential  equation 

dn(t)  =  —pn(t)  dt  +  a  dW (t) 

where  p  >  0  and  a  >  0  are  parameters  and  W (t)  denotes  the  Wiener  process.  Again,  the  solution  is 
provided  by 

n(t)  =  exp  (— p(t  —  to))n(to)  +  /  exp  (— p(t  —  s))a  dW(t) 

Jt0 
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In  this  formulation  a  >  0  and  p  >  0.  The  parameters  p  and  o,  together  with  the  initial  condition  n(0), 
completely  characterize  the  dynamics,  and  can  be  quickly  characterized  as  follows. 

i.  p  characterizes  the  velocity  at  which  such  trajectories  will  converge  to  zero; 

ii.  a  measures  the  amplitude  of  randomness  entering  the  system.  Higher  a  implies  more  randomness. 

iii.  o2 /2p  is  the  long  term  variance. 

We  note  that 

E[n(f)]  =  exp  {-p{t  -  t0))n(t0) 

2 

s )  =  Cov[n(,s),  n(t)]  =  E[(n(s)  —  E[n(s)])(n(£)  —  E[n(t)])]  =  —  exp(— p(s  +  £))(exp(2pmin(s,  t )  —  1) 

Ip 

<j2 

=  — (exp (—pit  —  s ))  —  exp(— p(t  +  s)))  provided  s  <  t. 

2  p 

For  large  p,  this  process  approximates  zero-mean  Gaussian  white  noise  with  covariance  (T2/2p  [139].  This 
model  is  sometimes  called  the  Singer  model  in  the  general  area  of  multiple  target  tracking  [38,  178].  Also, 
this  stochastic  process  can  be  included  in  the  system  dynamics  within  the  framework  described  by  equation 
(6.1.2)  with  white  noise. 


Second-Order  Gauss-Markov  Process 


Leonard,  Nievinski,  and  Born  [177]  suggest  that  stochastic  accelerations  in  the  perturbed  two-body  problem 
of  orbital  mechanics  can  be  modeled  as  second-order  Gauss-Markov  processes  (GMP2).  Following  Leonard 
et  ah,  a  scalar  GMP2  process  is  defined  by 


d2n(t.)  „  nit)  9  .  . 

-^  =  -2(w^r-wn(t)+aw(t)’ 

where  to  >  0,  S  >  0,  0  <  £  <  1,  w(t )  is  zero-mean  Gaussian  white  noise  with  unit  variance,  and  n(io), 
(,n^p ,  an(|  iijit)  are  mutually  independent.  Defining  3  =  oj\/l  —  C2,  the  autocovariance  function  for  n(t) 
is  found  to  be 


T>(t,  s )  =  E[(n(t)  -  E[n(t)])  ( n(s )  -  E[?r(s)])7  ] 


<7 


4  w3C 


3-Cw|t-*| 


cos  3\t  —  s|  +  sin  j3\t  —  s| 


By  reduction  of  the  second-order  SDE  for  n(t)  to  first-order  form  and  by  state  augmentation,  the  nonlinear 
SDE  with  a  GMP2  noise  process  can  be  reduced  to  one  (6.1.2)  in  which  the  noise  process  is  white. 
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6.2  Structural  and  Parameter  Uncertainty 

6.2.1  Structural  Uncertainty 

The  term  “structural  uncertainty”  refers  to  the  uncertainty  in  the  force  model  or  models  discussed  in  Chapter 
4  and  Section  3.3.  Mathematically,  “structural  uncertainty”  refers  to  Sf(x(t),  a ,  t)  in  the  equation 

a ,  t)  +  Sf(x(t ),  a,  t )  (6.2.1) 

wherein  one  actually  uses  the  model  (6.1.2)  instead  of  the  true  model  (6.2.1).  In  this  case,  the  term 
f(x(t),a,t )  +  Sf(x(t),a,t )  represents  the  true  forces  acting  on  the  RSO.  Thus  the  problem  is  to  esti¬ 
mate  df(x(t),  a ,  t)  or  to  account  for  the  uncertainty  by  a  stochastic  acceleration  such  as  that  modeled  as 
G(x(t),t)  n(t )  in  (6.1.2).  Here,  one  approach  is  to  make  use  of  the  Uhlenbeck-Ornstein  model  described 
in  Section  6.1.1  or  Vasicek  process.  The  parameter  in  the  Uhlenbeck-Ornstein  model  could  be  estimated  or 
the  parameter  could  be  subjugated  to  a  second  stochastic  differential  equation.  A  second  approach  to  this 
problem  is  to  estimate  the  unmodeled  dynamics  using  optimal  control  theory  as  discussed  in  Sections  3.3, 
6.6,  and  6.7.  Structural  uncertainties  are  usually  related  to  the  force  model  accelerations,  and  the  particular 
force  model  formulation.  While  many  of  the  force  models  have  become  very  robust,  the  specific  use  of 
each  model  can  sometimes  be  complicated  by  the  fact  that  little  or  no  information  regarding  the  actual  use 
is  available  to  an  outside  user.  For  example,  the  EGM-08  gravitational  model  is  extremely  accurate.  Yet,  if 
one  uses  a  12  x  12  field  and  another  uses  a  70  x  70  field,  they  will  get  very  different  answers.  Even  greater 
differences  would  arise  if  simulations  used  a  WGS-72  gravitational  model  while  another  used  the  EGM-08 
model. 

In  addition  to  the  force  model  itself,  the  input  data  parameters  (gravitational  coefficients,  solar  flux,  etc) 
can  be  equally  important  to  the  results  of  a  propagation.  Consider  the  solar  flux  parameters  that  are  required 
for  most  atmospheric  models.  Is  the  solar  flux  observed  or  adjusted  to  the  Earth’s  average  position  from  the 
Sun?  Is  the  81 -day  average  found  from  the  previous  81  days  for  which  there  is  measured  data,  or  from  40 
or  80  days  in  the  future  when  half  or  more  of  the  data  is  predicted,  and  therefore  much  less  reliable?  What 
are  the  sources  and  accuracy  of  the  predicted  data?  Answers  to  all  these  questions  have  never  been  collected 
into  a  standards  document. 

Several  standards  organizations  attempt  to  provide  a  means  for  users  to  exchange  information  (ISO, 
CCSDS,  etc).  While  there  is  rudimentary  information  in  many  of  these  standards  to  permit  a  user  to  ac¬ 
curately  reproduce  an  ephemeris  from  the  given  state  vector,  there  is  still  a  need  to  encapsulate  “enough” 
information  so  a  user  can  reproduce  an  ephemeris  from  the  given  state  information  within  a  specified  level 
of  accuracy  throughout  the  ephemeris.  Vallado  [259]  investigated  some  of  the  requirements  needed  to  en¬ 
sure  a  consistent  orbital  propagation  between  two  organizations.  The  conclusions  suggested  that  numerous 
additional  parameters  and  information  were  needed  beyond  a  simple  list  of  force  models.  An  example  is  the 
Conjunction  Data  Message  (CDM).  While  basic  force  model  information  is  included,  there’s  really  no  way 
to  accurately  propagate  the  given  states  because  one  barely  knows  which  force  model  (EGM-08,  MSIS-00, 
etc)  is  used,  let  alone  how  the  data  input  parameters  are  used  (interpolation,  last  average,  etc).  There  is  also 
no  information  about  how  to  propagate  the  covariance. 
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6.2.2  Parameter  Uncertainty  in  the  Equations  of  Motion 

Parametric  uncertainties  in  the  measurement  equations  and  dynamic  model  ((6.1.1)  and  (6.1.2)  or  (6.1.4) 
and  (6.1.3))  can  be  addressed  in  several  ways  depending  on  observability.  The  vector  of  parameters  a  S  Mp 
may  be  certain  or  uncertain.  Here  are  some  options  for  treating  these  parameters. 

a.  If  a  parameter  is  certain  or  known  significantly  below  the  remaining  uncertainties  in  the  system,  then  the 

parameter  may  be  considered  as  fixed. 

b.  If  a  parameter  is  included  in  the  orbit  determination  procedure  and  is  observable,  then  its  estimate  and  the 

uncertainty  in  the  estimate  is  available  at  epoch.  One  can  then  include  the  parameter  a  in  the  dynam¬ 
ical  system  as  ^  =  0  and  propagate  its  uncertainty  through  the  differential  equation  to  some  future 
time.  One  might  also  consider  in  this  treatment,  the  Schmidt- Kalman  filter  [235],  which  addresses  the 
statistical  cross  correlation  between  the  uncertain  parameters  and  the  state. 

c.  The  case  in  which  some  of  the  system  parameters  are  not  observable  is  more  complex.  One  possibility 

is  to  use  batch  estimation  combined  and  the  singular  value  decomposition  [112]  to  determine  those 
parameters  or  roll-ups  that  are  observable  and  those  that  are  not.  If  a  parameter  is  observable,  then 
it  can  be  estimated  and  treated  as  above.  In  case  a  parameter  is  not  observable,  it  is  often  treated 
probabilistically  by  assuming  a  uniform  probability  density  function  on  an  interval  of  a  parameter  and 
approximating  the  uniform  distribution  with  a  Gaussian  sum  [61],  by  polynomial  chaos  (see  Section 
7.1 1),  or  deterministically  as  a  systemic  uncertainty  by  sampling  techniques  [204,  229]. 

d.  It  is  sometimes  possible  to  construct  an  analytical  model  to  approximate  the  effect  of  the  uncertainty  in 

a  set  of  parameters.  This  can  be  important  when  the  number  of  uncertain  parameters  is  very  large  and 
often  weakly  or  non-observable  such  as  an  uncertain  gravitational  field  which  may  contain  thousands 
of  coefficients.  While  inclusion  of  the  uncertain  parameters  in  the  state  is  computationally  prohibitive, 
it  is  possible  to  obtain  a  tractable  approximation  to  the  associated  growth  in  orbit  uncertainty  [292] . 

6.2.3  Parameter  Uncertainty  in  the  Measurement  Equation 

Parameters  in  the  measurement  equations  are  generally  called  sensor  biases,  navigation  errors,  and  time 
biases,  and  are  estimated  using  truth  or  fuzzy  truth  objects  as  training  data.  The  different  biases  are  ad¬ 
dressed  more  specifically  in  Chapter  5.  The  determination  of  these  parameters  is  normally  part  of  sensor 
calibration.  Some  of  these  parameters  may  be  observable  or  unobservable  while  rollups  may  be  observable 
or  unobservable.  Again,  this  can  be  determined  using  the  singular  value  decomposition  [112]. 

6.3  The  Filtering  Problem  for  Continuous  and  Discrete  Dynamics 

Given  an  initial  probability  density  function  (PDF)  describing  the  uncertainty  in  the  state  of  an  object,  say, 
p  =  p(x,  tk_i\Zk_i),  the  PDF  at  a  future  time  p  =  p(x,  t\Z^_i)  for  t  >  t^-i  is  governed  by  the  Fokker- 
Planck-Kolmogorov  equation  (FPKE)  [139]. 

%  =  -V£(p/)  +  ^tr[V*v£(pGQGT)],  (6.3.1) 
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where  Vx  is  the  gradient  with  respect  to  x  viewed  as  a  column  operator  and  n{t)  is  assumed  to  be  white 
noise  with  power  spectral  density  Q.  This  is  sometimes  called  the  prediction  step  or  time  update.  Given  a 
new  sensor  report/measurement  zk  at  time  tk,  the  measurement  update  is  obtained  from  Bayes’  rule  using 
the  measurement  density  p(zk \xk)  and  the  predicted  density  via 


p{x(tk),tk  |  Zk) 


p{zk  |  x(tk))  p{x(tk)  |  Z^- 1) 

f p(zk  |  x (tk))  p(x(tk)  |  Zfc_i)  dx(tk) ' 


(6.3.2) 


Analytical  solutions  to  the  FPKE  and  to  the  prediction  and  correction  equations  in  Figure  6.3.1  are  gen¬ 
erally  intractable  and  are  only  known  in  a  few  restrictive  cases.  Conventional  approaches  for  the  numerical 
solution  of  this  equation  suffer  from  the  “curse  of  dimensionality”;  however,  sparse  grid  methods  for  solving 
the  FPKE  show  promise  for  a  real  time  solution  and  are  still  very  much  a  research  problem  [161]. 

For  nonlinear  and  non-Gaussian  models,  approximate  or  suboptimal  algorithms  are  utilized  for  the 
Bayesian  state  estimator.  The  most  often  used  approximate  algorithms  are  based  on  the  extended  Kalman 
filter  (EKF),  unscented  Kalman  filter  (UKF),  and  perhaps  the  second  order  extended  Kalman  filters.  These, 
however,  represent  the  propagation  of  the  mean  and  covariance  and  do  not  directly  address  the  propaga¬ 
tion  of  non-Gaussian  processes  encountered  in  astrodynamics.  Due  to  the  need  to  propagate  uncertainties 
over  extended  periods  of  time  in  the  absence  of  measurement  updates,  higher-order  cumulants  (e.g.,  skew¬ 
ness,  excess  kurtosis)  can  become  non-negligible  and  must  be  accounted  for  in  order  to  achieve  uncertainty 
consistency. 

In  a  recursive  filtering  approach  for  discrete  dynamics  (3),  measurement  data  is  processed  sequentially, 
rather  than  as  a  batch.  Given  the  initial  density  of  the  state  p(x o)  =  p(xo\Zo),  the  PDF  p(xk\Zk)  is 
obtained  recursively  in  two  stages,  namely  prediction  and  correction,  as  illustrated  in  the  flowchart  of 
Figure  6.3.1.  The  former  is  obtained  from  the  transitional  density  p(xk\xk_i)  in  conjunction  with  the 
Chapman-Kolmogorov  equation  defined  in  the  prediction  step  box  of  Figure  6.3.1. 


Figure  6.3.1  Predictor-corrector  step  for  the  recursive  Bayesian  state  estimator 


The  term  p(zk\Zk_i)  in  the  denominator  of  the  correction  step  is  called  the  prediction  error  and  appears  in 
the  likelihood  ratios  for  scoring  an  assignment  of  a  report  (i.e.,  zk)  to  a  track  (see,  for  example,  Poore  [224]). 
Thus,  its  accurate  evaluation  is  critical  for  correct  data/track  association  (correlation). 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


61 


CHAPTER  6.  THE  INVERSE  PROBLEM:  A  BRIEF  REVIEW  OF  ES TIMATION  FORMULATIONS 
6.4.  FILTER-SMOOTHER  AND  THE  RELATIONSHIP  TO  BATCH l BWMM'E  SURVEILLANCE 

6.4  Filter-Smoother  and  the  Relationship  to  Batch  Estimation 


There  are  several  filter-smoothing  algorithms  (e.g.,  fixed-interval,  fixed  point,  and  fixed  lag)  in  which  one 
determines  the  probability  density  function  p(x(t k)  \  Zn )  wherein  in  N  >  k.  A  discussion  of  these  filter- 
smoother  algorithms  can  be  found  in  the  books  by,  for  example,  Gelb  [96],  Sarkka  [233],  Crassidis  and 
Junkins  [55]  and  Tapley,  Schutz,  and  Born  [254].  The  relationship  between  the  extended  Kalman  filter/s¬ 
moother  and  batch  estimation  (with  or  without  process  noise)  has  been  established  in  the  literature  starting 
with  the  work  of  Bell  and  Cathey  [29],  Bell  [28],  Bertsekas  [33]  and  summarized  in  the  2012  review  paper 
by  Humpherys,  Redd,  and  West  [132],  Essentially,  a  Gauss-Newton  step  is  equivalent  to  a  filter-smoother 
pass  as  shown  in  this  review  paper  [132]  and  the  paper  by  Bell  [28]. 

The  astrodynamics  surveillance  community  makes  extensive  use  of  batch  estimation  and  filter-smoothers. 
Batch  estimation  is  often  solved  by  a  sequence  of  Gauss-Newton  steps;  however,  most  nonlinear  least 
squares  algorithms  add  a  globalization  method  such  as  a  line  search  or  a  trust  region  method  to  assist  with 
convergence. 

For  the  filter/smoother  solution,  the  McReynoIds  test  [293]  was  developed  to  test  the  consistency  between 
the  filter  and  smoother.  This  in  a  way  helps  ensure  a  consistent  performance.  The  filter-smoother  consistency 
test  is  applied  by  testing  that  the  difference  between  the  filtered  state  (Xj)  and  smoother  state  (Xs)  is 
normally  distributed  in  n  dimensions  (assuming  n  is  the  size  of  the  state-difference  vector)  with  a  covariance 
equal  to  the  filter  covariance  subtracted  from  the  smoother  covariance  [201].  The  test  may  be  applied  to 
the  entire  estimation  state  or  any  subset  thereof.  In  practice  the  filter-smoother  consistency  test  is  often 
applied  to  individual  components  of  a  state,  which  often  provides  more  focused  diagnostic  information. 
Due  to  correlations  between  the  filtered  and  smoothed  solutions,  it  is  also  common  practice  to  plot  this 
metric  as  a  function  of  time  evaluated  against  confidence  bounds  which  depend  on  the  dimension  of  the 
sub-state  involved  in  the  computation  of  the  metric  as  opposed  to  applying  formal  distribution  tests.  For 
example,  filter-smoother  consistency  is  generally  claimed  for  the  single  state  case  when  this  metric  stays 
within  pm3cr  over  the  fit  interval. 

outside  expected  bounds  is  typically 


Significant  excursion  of  the  test  metric  ^Consistency 


_ 


Xf  —  Xg 


the  result  of  an  incorrect  dynamical  model,  incorrect  measurement  statistics  or  the  processing  improper 
measurements.  Typical  examples  of  problems  are  incorrect  ballistic  or  solar  pressure  coefficient  characteri¬ 
zations,  un-modeled  out-gassing  or  momentum  dumps,  tracking  data  from  an  improperly  configured  ground 
station  and  mis-tagged  observations. 

Ay  an  aside,  given  the  relationship  between  Gauss  Newton  steps  in  batch  estimation  and  the  above 
consistency  test  for  the  filter-smoother,  it  would  be  interesting  to  investigate  the  relationship  between  this 
consistency  test,  the  Gauss  Newton  steps  and  the  use  of  a  globalization  method  for  nonlinear  least  squares 
such  as  a  line  search. 


6.5  The  Consider  or  Schmidt-Kalman  Filter 

The  Schmidt-Kalman  filter  is  a  modification  of  the  Kalman  filter  that  addresses  a  state  and  parameters  in 
the  dynamics  and  measurement  equations  wherein  the  parameters  are  assumed  to  be  Gaussian  distributed 
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with  known  means  and  covariances.  Rather  than  estimating  these  parameters  as  part  of  a  Kalman  filter,  the 
Schmidt-Kalman  filter  builds  the  statistics  of  the  parameters  into  the  state  of  the  object  and  its  statistics.  Here 
is  an  example.  Assume  that  the  system  parameters  in  both  the  dynamics  and  the  measurement  equations  have 
been  estimated  with  the  resulting  estimates  being  Gaussian  random  variables.  The  Schmidt-Kalman  filter 
then  accounts  for  the  cross-correlation  of  between  estimated  parameters  and  the  state  thereby  modifying  the 
covariance  of  the  state.  The  primary  advantage  of  utilizing  the  Schmidt-Kalman  filter  instead  of  increasing 
the  dimensionality  of  the  state  space  is  the  reduction  in  computational  complexity.  This  can  enable  the 
use  of  filtering  in  real-time  systems.  Another  usage  of  Schmidt-Kalman  occurs  when  residual  biases  are 
unobservable;  that  is,  the  effect  of  the  bias  cannot  be  separated  out  from  the  measurement.  . 

The  Schmidt-Kalman  filter  [235]  is  also  presented  in  the  book  by  Jazwinski  [139]  and  more  recent 
articles  by  Zanetti  and  D  Souza  [297],  Stauch  and  Jah  [249]  combine  the  unscented  Kalman  filter  with  the 
Schmidt-Kalman  filter  into  a  unscented  Schmidt-Kalman  filter.  A  different  approach  to  a  consider  filter  can 
be  found  in  the  book  by  Tapley,  Schutz,  and  Born  [254]. 

6.6  Batch  Estimation  with  Process  Noise  in  the  Continuous  Case 

A  formulation  of  the  estimation  problem  in  the  case  of  continuous  dynamics  with  process  noise  can  be  found 
in  the  book  by  Jazwinski  [139].  When  the  measurement  noise  sequence  is  white  zero  mean  Gaussian  with 
uk  ~  N{ 0,  Rfc)  and  the  process  noise  is  white  zero  mean  Gaussian  w(t)  ~  N( 0,  Q(t)),  xq  ~  N(xq..  Po), 
and  these  are  independent,  then  the  following  is  often  used: 

1  T  _i 

Minimize  (x  w)J(x,w)  =  ~(x0  -  x0)  P„  (*o  -  *o)  (6.6.1) 

1  N 

+  2  (Zk  ~  hk(x(tk)))^k1  {*k  -  hk(x(tk))) 

Z  k= 1 

1  rtN 

+  -  /  w(t)t  Q-1  (t)w(t)cIt 

2  Jt0 

Subject  To:  dx  =  f(x,a,w(t),t)dt,  to<t<tN  (6.6.2) 

to  characterize  the  state  and  its  uncertainty.  (The  case  of  N  =  1  corresponds  to  sequential  processing.)  Note 
that  any  of  the  process  noises  in  Section  6.1.1  can  be  treated  in  this  way  via  state  augmentation.  If  one  thinks 
of  this  formulation  from  a  controls  viewpoint,  a  feature  is  that  the  term  f{x.  a.  w(t),t)~  f{x,  a,  0,  t  )  might 
now  be  interpreted  as  the  model  bias  or  model  inadequacy  in  the  model  dynamics  f(x,  a,  0,  t)  discussed 
in  Section  6.2.1.  A  difference  in  the  treatment  of  w  as  a  control  instead  of  noise  is  that  w  is  determined  a 
posteriori  as  part  of  the  solution  rather  being  specified  a  priori  in  the  case  of  process  noise.  This  optimal 
control  approach  could  also  assist  with  the  design  of  the  process  noise  discussed  in  Section  6.1.1.  Here  are 
two  observations.  In  both  cases,  the  choice  of  the  covariance  matrix  Q(t)  is  the  nontrivial  part  of  the  design 
problem  and  can  significantly  impact  the  results.  The  data  association  is  assumed  to  be  correct;  otherwise, 
eiToneous  conclusions  can  follow  from  a  choice  of  a  control  that  allows  one  to  “connect”  measurements 
that  emanate  from  different  objects.  Finally,  another  discussion  of  control  approaches  is  also  contained  in 
Section  3.3. 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


63 


CHAPTER  6.  THE  INVERSE  PROBLEM:  A  BRIEF  REVIEW  OF  ES TIMATION  FORMULATIONS 
6. 7.  BATCH  ESTIMATION  WITH  PROCESS  NOISE  IN  THE  DISCMWE  IN\SE\CE  SURVEILLANCE 


The  corresponding  problem  without  process  noise  or  control  w  is 


Minimize  J(xq) 


7)  (*0  -  *o)  1  P0  1  (*0  -  *o)  + 


2  iZk  hk{x(tk)))R-k  {Zk 

z  k= 1 


Subject  To: 


dx 

dt 


f(x,a,t),  t0  <  t  <  tN,  x(t0)  =  x0 


hk(x(tk))) 


(6.6.3) 

(6.6.4) 


wherein  the  only  independent  variable  is  xq.  For  initial  orbit  determination,  the  first  term  is  generally  absent. 
In  the  astrodynamics  community,  the  solution  method  for  this  problem  is  called  differential  corrections. 
When  process  noise  is  absent,  the  covariance  of  the  state  can  “collapse;”  however,  this  difficulty  is  often 
mitigated  with  the  use  of  a  fading  memory  process. 


6.7  Batch  Estimation  with  Process  Noise  in  the  Discrete  Case 

Given  the  discrete  dynamical  system  (6.1.3)  and  measurements  (6.1.4),  the  objective  of  initial  orbit  determi¬ 
nation  (IOD)  and  batch  estimation  is  to  obtain  a  representation  of  the  (joint)  posterior  PDF  p(x  o , . . . ,  ®  jv  |  Zn), 
where  xk  denotes  the  dynamical  state  of  the  system  at  time  tk,  and  to  extract  meaningful  statistics  (e.g., 
mean,  covariance)  from  it  in  a  consistent  manner.  It  is  assumed  that  the  state  evolves  according  to  the 
discrete-time  model 


Xk  =  fk- i(®fc-i,  t/tfc-i)  =  /fc_ i(®fc_i)  +  tWfc-1,  (k  >  1)  (6.7.1) 

instead  of  the  representation  (3).  Appealing  to  Bayes’  rule  and  the  above  assumptions,  the  joint  posterior 
PDF  is  derived  in  Jazwinski  [139,  §5.3]  and  is  found  to  be 

N  N 

p{x 0, . .  .,xN\Zm)  =  cpo(x0)  W_Pwk(xk  -  fk_ i(ajfc_i))  Y\.Pvk(zk  -  hk(xk)),  (6.7.2) 

k= 1  k=  1 

where  c  is  a  normalizing  constant,  and  po  is  the  prior  PDF  of  the  state  xq  at  time  to-  Further,  in  (6.7.2),  the 
pWk  and  pVk ,  for  k  =  1 .....  Ar,  are  the  respective  PDFs  of  the  process  and  measurement  noise  processes.  In 
practice,  they  are  often  assumed  to  be  Gaussian  with  zero  mean  and  covariances  of  Q/  and  R/.,  respectively. 

The  posterior  PDF  (6.7.2)  is  the  complete  description  of  the  uncertainty  of  the  state  at  each  of  the 
measurement  times.  In  practice,  a  finite  dimensional  representation  of  the  uncertainty  is  sought.  Thus,  the 
emphasis  of  the  batch  estimation  problem  is  on  how  statistical  information  can  be  extracted  from  (6.7.2) 
consistently  and  accurately.  Nonlinear  optimization  theory  provides  a  framework  for  computing  the  modal 
trajectory  or  maximum  a  posteriori  (MAP)  estimate  of  (6.7.2).  For  a  Gaussian  prior  with  xq  ~  N(a 5o,  Po) 
and  Gaussian  noise  processes  as  described  above,  the  modal  trajectory  is  obtained  by  solving  the  least 
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squares  or  batch  problem 

(*o,  ■  •  -,xn)map  =  arg  max  p(x0, . .  .,xN\ZN) 

XO  ,...,£CJV 

^  ^  m  i  N  (6.7.3) 

=  arg  min  -||*0  -  ®o||p-i  +  -  V  \\xk  -  fk- i(®fc-i)||o-i  +  -  V  || zk  -  hk(xk) ||;L  i. 
xo,...,xN  z  Z  z '  ^fc-1  Z  z '  L^k 

k=  1  k= 1 

If  the  matrix  depends  on  then  one  needs  to  add  the  term  Y2k=i  hi  det(27rQ/,._  ]  [xk- 1 )) . 

Methods  for  solving  nonlinear  least  squares  problems,  such  as  Gauss-Newton,  full  Newton,  and  quasi- 
Newton  updates,  along  with  globalization  methods  such  as  line  search  and  trust  region  methods  including 
Levenberg-Marquardt  [88],  are  efficient  and  mature  and  will  not  be  discussed  further  here.  In  the  astrody- 
namics  community,  such  solution  techniques  are  called  differential  correction  methods.  In  any  nonlinear 
least  squares  problem  such  as  (6.7.3),  one  must  provide  the  solver  a  starting  guess  in  order  to  initiate  the 
differential  correction  method.  This  is  the  initial  orbit  determination  (IOD)  problem.  In  the  case  of  mea¬ 
surement  data  from  a  single  radar  or  EO  sensor,  a  first  estimate  can  be  obtained  using  the  classical  methods 
of  Lambert  or  Gauss  (see,  for  example,  [260]).  Additionally,  for  angle-only  observations,  a  recent  algorithm 
due  to  Gooding  [103]  has  shown  promise  for  IOD  scenarios  involving  both  ground-based  and  space-based 
EO  sensors  [263].  An  improvement  can  be  found  in  the  recent  work  of  Casotto  [46],  Finally,  it  is  im¬ 
portant  to  stress  that  batch  estimation  is  generally  employed  using  a  sliding  window  rather  than  using  all 
measurements  over  all  time. 

Finally,  one  can  also  consider  this  discrete  formulation  within  the  context  of  a  discrete  optimal  control 
approach  discussed  in  Section  6.6. 

6.8  The  Use  of  Empirical  Covariance  Matrices 

The  above  discrete  least  squares  formulations  for  IOD  or  OD  in  equations  (6.6.3)  for  continuous  dynamics 
and  no  process  noise  or  discrete  dynamics  (6.7.3)  with  or  without  process  noise  can  all  be  formulated 
abstractly  as  solving  for  x  from  the  equation 


f(x)  =  0  +  e 

wherein  e  ~  N( 0,  W),  i.e.,  is  a  zero  mean  Gaussian  with  covariance  W,  f  :  Mm  — »•  is  a  smooth 

function  and  W  £  Mmxm  is  a  symmetric  positive  definite  matrix.  The  unknown  x  =  xq  for  the  formulation 
(6.6.3)  or  x  =  (xq.  x  .x  ;.y )  for  the  discrete  problem  6.7.3  with  process  noise.  These  problems  are 
normally  formulated  as 

Minimizea;||/(x)||^v_1  (6.8.1) 

Given  a  minimizer  x,  one  generally  writes  the  maximum  likelihood  estimate  of  the  covariance  matrix  as 

P  =  (J(£c)TW“1J(a;))_1 

where  J(a:)  =  Dxf(x )  is  the  Jacobian  matrix  of  f(x). 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


65 


CHAPTER  6.  THE  INVERSE  PROBLEM:  A  BRIEF  REVIEW  OF  ES TIMATION  FORMULATIONS 
6. 8.  THE  USE  OF  EMPIRICAL  COVARIANCE  MATRICES  USED  IN  SPACE  S  URVEILLANCE 

6.8.1  An  Empirical  Covariance  Matrix  for  Batch  Processing 

An  empirical  covariance  has  been  derived  by  Frisbee  [155].  In  view  of  the  equivalence  between  batch 
and  the  filter/smoother,  Frisbee’s  empirical  covariance  applies  both  to  the  filter/soother  as  well  as  the  batch 
processing.  Extensions  to  the  iterated  EKF  and  other  filters  may  also  be  possible. 

Here  is  a  synopsis  of  this  work  [155].  Matrices  originating  through  the  usual  estimation  techniques 
(batch,  sequential)  are  theoretical  state  error  covariance  matrices  representing  only  the  effect  of  assumed 
measurement  uncertainties.  Techniques  exist  in  both  batch  and  sequential  methods  for  adding  a  so-called 
process  noise.  (Notwithstanding  the  following  description,  process  noise  actually  has  a  more  simple  ex¬ 
planation.)  Typically  such  noise  is  determined  empirically  through  prior  analysis  of  the  performance  of 
any  specific  estimation  process.  Any  process  noise  added  is  intended  to  represent  the  historical  averaged 
effect  of  unmodeled  error  sources.  The  effectiveness  of  process  noise  is  dependent  upon  the  consistency 
of  the  noise  sources.  This  consistency  should  be  both  within  the  analysis  leading  to  the  determination  of 
the  process  noise  as  well  as  between  the  future  solutions  to  which  it  as  applied  and  the  past  history  from 
which  it  is  derived.  With  regards  to  the  state  error  covariance  matrix,  classical  process  noise  has  no  ability 
to  adequately  account  for  within  solution  model  or  measurement  noise  variations.  Neither  can  the  classical 
process  noise  account  for  spontaneous  changes  in  the  measurement  noise  or  dynamic  model. 

An  alternative  is  the  empirical  state  error  covariance  matrix  which  is  created  in  such  a  way  as  to  include 
all  errors  reflected  in  the  measurement  residuals.  The  empirical  state  error  covariance  matrix  includes, 
without  regard  to  whether  the  source  might  be  known  and  anticipated  or  unknown  and  spontaneous,  all 
errors  existing  over  the  estimation  fit  interval  and  reflected  in  the  measurement  residuals.  The  empirical 
error  covariance  matrix  also  represents  each  solution  uniquely  and  individually.  That  is  if  a  condition  existed 
temporarily  and  in  such  a  way  so  as  to  only  affect  a  satellite  or  some  group  of  satellites  during  a  short  interval 
of  time  (fit  interval,  day,  hour,  observation)  the  empirical  error  covariance  matrix  has  the  ability  to  reflect 
the  changes  that  occurred.  This  is  true  even  if  the  effects  are  not  recognized  to  have  occurred  at  the  time. 

The  batch  estimation  empirical  state  error  covariance  matrix,  as  presented  by  Frisbee  [155],  is  just  the 
final  part  of  the  standard  sample  data  problem.  That  problem  gives  rise  to  various  statistics  such  as:  the 
average  (estimate  of  the  population  mean),  empirical  covariance  (estimate  of  the  population  covariance), 
and  empirical  covariance  of  the  average  (estimate  of  the  uncertainty  of  the  average  with  respect  to  the 
population  mean).  Traditionally,  estimation  has  only  addressed  the  estimate  of  the  mean  and  the  theoretical 
value  of  the  covariance  of  the  average  with  respect  to  the  mean.  Batch  estimation  is  a  simple  application 
of  a  standard  sample  data  problem  in  frequentist  statistics.  All  of  the  usual  rules  of  that  discipline  apply. 
As  an  example,  the  state  error  covariance  matrix  is  a  sample  covariance  and  thus  is  inherently  uncertain. 
However  there  are  specific  paths  within  statistics  to  address  such  uncertainty.  It  might  be  argued  that  the 
current  theoretical  error  covariance  matrices  are  not  uncertain.  That  is  correct.  However,  those  matrices  are 
just  simply  wrong  because  they  leave  out  all  error  sources  except  the  assumed  measurement  uncertainties, 
which  could  themselves  be  wrong. 
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6.8.2  Use  of  an  Empirical  Covariance  for  Propagation 

One  of  the  challenges  in  using  empirical  covariances  in  estimating  the  uncertainty  in  orbit  determination  is 
that  the  data  is  usually  sparse.  Dense  tracking  data  situations,  such  as  that  possible  from  GPS  receivers  in  a 
LEO  orbit  or  continually  telemetered  GEO  communication  satellites,  while  increasing  in  commonality,  are 
still  the  exception  rather  than  the  rule,  as  most  objects  in  space  are  inactive,  and  of  those  that  are  active, 
data  is  usually  restricted  to  owner-operators.  The  time  scale  required  to  collect  enough  data  for  a  traditional 
empirical  covariance  would  likely  span  multiple  orbit  revolutions,  if  not  multiple  days,  over  which  short- 
periodic  or  transient  effects  could  be  lost  in  the  estimate.  As  noted  in  the  previous  section,  the  standard 
covariance  calculations  are  based  on  a-priori  knowledge  of  measurement  error  (reflected  in  the  measurement 
weighting  matrix)  and  modeling  error  (most  often  modeled  in  the  form  of  some  flavor  of  process  noise), 
which  are  rarely  a  precise  description  of  the  error  in  the  orbit.  However,  they  do  include  most  of  the  dominant 
sources  of  error,  and  their  propagation  includes  many  of  the  short  periods  and  transients  that  might  be  lost 
if  one  were  averaging  over  a  sparse  data  environment.  Thus,  there  is  a  benefit  to  simply  correcting  error  in 
existing  covariance  using  empirical  methods  rather  than  using  empirical  methods  alone. 

The  development  by  Cerven  [47,  48]  analyzes  the  error  in  the  covariance  through  normalization  of  the 
data  by  the  a-priori  covariance.  Let  the  true  covariance  P/  of  x  e  M"  be  defined  relative  to  the  predicted 
covariance  P?j  via  a  normalized  covariance  Pn  such  that 

Pt  =  (VpDp)Pn(VpDp)T, 

where  D;,  and  Vp  are  the  matrices  of  eigenvalues  and  eigenvectors,  respectively,  of  the  predicted  covari¬ 
ance  Pp.  This  normalized  covariance  then  encapsulates  the  error  in  the  predicted  covariance.  To  solve 
this  problem,  let  us  compare  predicted  states  xp  and  covariances  Pp  against  measurements  yrn  e  Mm  with 
covariance  Pm,  where  measurement  space  is  mapped  to  state  space  such  that  y  =  f(x)  (with  lineariza¬ 
tion  H(aj)  =  Dxf(x)).  The  difference  in  the  measurement  to  the  mapped  predicted  state  is  Ay  with  a 
covariance  P,/  such  that 


Ay  =  f{x)  -  ym, 

Pd  =  ¥Pn¥T  +  Pm  (\Er  =  HVpDp) ,  (6.8.2) 

P ty  =  n^T  =  P d~  Pm, 

where  Pty  is  the  mapping  of  P/  to  the  measurement  space  and  can  be  sampled  by  a  single  Ay  such  that 

P ty  =  &yAyT  -  Pm 

Letting  pty  and  pn  be  vector  representations  of  the  upper-triangular  matrices  of  P/y  and  Pn,  respec¬ 
tively,  one  can  use  creative  algebra  to  turn  equation  (6.8.2)  into  the  following  linear  equation. 

Pty  =  &(^)Pn 
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which,  through  sufficient  sampling  of  the  measurement  data  with  the  assumption  that  the  matrix  P„  is 
effectively  constant  across  the  time  spanned  by  the  measurements  in  question,  can  be  solved  by  a  simple 
linear  least  squares  method.  Furthermore,  this  can  be  mapped  directly  to  the  traditional  sample  covariance 
calculation  when  measurements  are  full  rank. 

As  this  estimate  of  the  normalized  matrix  Pn  is  based  on  sampling,  confidence  bounds  on  the  errors  in 
this  matrix  can  be  estimated  using  sample  distribution  theory  as  well,  which  can  be  calculated  in  terms  of 
the  emors  on  the  variances  and  correlation  factors  of  this  matrix.  The  sampling  distribution  of  the  variance 
takes  the  form  of  a  gamma  distribution  dependent  on  the  actual  standard  deviation  of  the  distribution  and  the 
number  of  samples,  while  the  sample  distribution  of  the  bivariate  correlation  coefficients  follows  Pearson's 
r-distribution  and  can  be  mapped  to  an  (approximately)  Gaussian  distribution  via  the  Fisher  transformation. 

The  sensitivity  of  the  probability  of  collision  to  these  confidence  bounds  can  be  found  by  finding  the 
minimums  and  maximums  over  the  range  of  variances  and  correlation  factors,  a  basic  parameter  optimiza¬ 
tion  problem. 

6.9  Particle  Filters 

Particle  filters  are  sequential  Monte  Carlo  methods  based  on  based  on  the  representation  of  an  arbitrary 
PDF  by  a  set  of  independent  and  identically  distributed  random  samples,  which  can  be  applied  to  any  state- 
space  model,  and  which  generalize  the  traditional  Kalman  filtering  methods  to  nonlinear  and  non-Gaussian 
problems  [21,  73,  228].  These  methods  are  used  in  space  object  tracking,  e.g.,  [174,  194,  241]. 

6.10  Nonlinear  Filtering  Using  Sparse  Grid  Methods 

Conventional  approaches  for  the  numerical  solution  of  the  Fokker-Planck-Equation  suffer  from  the  curse  of 
dimensionality  and  are  therefore  not  applicable  in  higher  dimensions.  Sparse  grid  methods  for  solving  the 
Fokker-Planck-Equation  show  promise  for  a  real  time  solution  and  are  still  very  much  a  research  problem 
[161]. 

6.11  FISST  Based  Methods 

Multi-target  filters  based  on  Finite  Set  STatistics  (FISST)  provide  a  method  for  estimating  a  multi-target 
state  when  given  a  set  of  observations  [187,  188].  As  described  in  Section  3.2,  data  association  based  on 
MHT/MFA  or  JPDA  provides  a  means  for  updating  a  single-target  state  when  given  a  method  for  data  as¬ 
sociation.  Instead,  FISST-based  algorithms  include  the  data  association  process  as  a  fundamental  part  of  a 
given  multi-target  filter.  Such  methods  leverage  Random  Finite  Sets  (RFS)  to  estimate  the  probability  den¬ 
sity  function  (PDF)  of  the  multi-target  state,  which  provides  an  estimate  (with  uncertainty)  in  the  number  of 
objects  and  their  states.  FISST  provides  the  mathematical  tools  for  characterizing  and  analyzing  uncertainty 
in  the  estimated  multi-target  state,  which  is  consistent  with  standard  probability  theory  [276]. 

In  the  RFS  approach,  the  multi-target  state  Xy.  and  observation  Zy.  at  time  ty  are  represented  by  the  finite 
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sets 


Xk  =  {xi,x2,  ■  •  •  ,xn}  C  X  (6.11.1) 

Zk  =  {z!,z2,  •  •  •  ,zm}  C  Z  (6.11.2) 

where  X  and  Z  denote  the  single-target  state  and  measurement  spaces,  respectively.  For  example,  if  there  are 
two  targets  with  states  x\  and  x2,  then  Xk  =  {xi,x2}  and  the  estimated  number  of  objects  is  the  number  of 

elements  in  Xk.  For  a  single  sensor  scan,  such  as  a  single  CCD  image,  there  may  be  multiple  measurement 

vectors  zi, ,  zrn.  The  measurement  Zk  is  then  the  set  of  these  observation  vectors  at  time  tk.  Note  that 
the  \Xk\  =  n  does  not  necessarily  equal  \Zk\  =  rn  due  to  missed  detections  and  clutter.  Each  single-target 
state  x  is  assumed  to  be  a  random  vector  with  realization  Xi  £  X.  The  Bayes  multi-target  filter  is  then  an 
analog  of  the  single-target  Bayes  filter  (see  6.3)  to  estimate  Xk  when  given  a  measurement  Zk.  Such  filters 
includes  a  framework  to  model  target  birth  (analogous  to  IOD  in  astrodynamics),  target  survival  probability, 
spawning  of  new  objects,  and  probability  of  detection.  Such  models  will  vary  with  application  and  required 
fidelity. 

Several  forms  of  the  FISST-based  filter  exist  to  satisfy  variations  in  system  requirements  and  computa¬ 
tion  considerations.  The  FISST  filter  provides  a  full  PDF  of  the  multi-target  state,  but  is  intractable  for  a  large 
number  of  targets  [187]  .  This  motivated  the  development  of  principled  approximations  of  the  FISST  filter. 
The  most  well  studied  of  these  include  the  Probability  Hypothesis  Density  (PHD)  filter  [185,  275,  276], 
the  Cardinalized  Probability  Hypothesis  Density  (CPHD)  filter  [186,  279],  and  various  forms  of  the  multi- 
Bernoulli  filter  (e.g.,  see  [227,  277,  278]).  The  PHD  filter  estimates  the  density  of  objects  in  the  single-target 
space,  but  does  not  include  knowledge  of  target  identity.  To  improve  stability  in  the  PHD  filter,  the  CPHD  fil¬ 
ter  augments  the  density  function  with  an  estimated  cardinality  distribution,  i.e.,  a  probability  mass  function 
describing  the  number  of  targets.  Alternatively,  the  multi-Bernoulli  filters  estimate  Xk  based  on  a  param¬ 
eterization  {(rj,pj(a:))}^1  where  r,  is  the  probability  of  existence  for  the  i-th  target  and  pi(x)  is  the  state 
PDF  Such  a  parameterization  allows  for  estimating  information  on  specific  targets  instead  of  a  more  general 
target  density  function  [278].  A  general  description  of  finite  set  statistics  and  the  various  FISST-based  filters 
may  be  found  in  [187]  and  [188]. 

Applications  of  RFS-based  multi-target  filters  for  tracking  orbital  debris  have  been  demonstrated  in 
recent  literature.  This  includes  the  FISST  filter  for  at  most  three  targets  [133,  133,  134],  the  PHD/CPHD 
filter  for  up  to  hundreds  of  objects  [51,  95,  150,  195],  and  the  labeled  Multi-Bernoulli  filter  (LMB)  [152]. 
The  fundamental  forms  of  the  PHD-based  filters  maintain  no  identifying  information,  e.g.,  a  catalog  number. 
The  LMB  filter  described  in  [227]  and  applied  to  tracking  orbital  debris  in  [152]  provides  a  means  for 
maintaining  knowledge  of  target  identity  in  the  context  of  the  FISST-based  filters. 

6.12  Recommendations 

Many  of  the  estimation  methods  are  surveyed  above.  Virtually  all  of  the  current  estimation  techniques  are 
being  used  to  estimate  the  state  and  uncertainty  of  resident  space  objects.  Here  are  some  final  comments 
and  recommendations. 
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1 .  A  set  of  benchmark  test  cases  similar  to  those  proposed  in  Chapter  9  for  assessing  covariance/uncer¬ 
tainty  realism  during  propagation  should  be  developed  for  assessing  uncertainty/covariance  realism  at 
epoch. 

2.  In  addition  to  state  estimation,  the  uncertain  parameters  in  the  dynamics  and  sensor  biases,  navigation 
errors,  and  time  biases  in  the  measurement  equation  should  all  be  considered  and  potentially  estimated 
if  observable.  The  latter  are  generally  determined  as  part  of  the  calibration  process  and  the  former 
as  part  of  the  orbit  determination  process.  Finally,  a  consider  filter  might  be  used  to  improve  the 
estimation  of  uncertainty  due  to  statistical  cross  correlation  between  the  uncertain  parameters  in  the 
dynamics  and  measurement  equations. 

3.  The  astrodynamics  community  is  divided  on  the  use  of  process  noise  or  stochastic  acceleration;  how¬ 
ever,  process  noise  is  one  technique  that  can  account  for  model  mismatch  and  parameter  uncertainty, 
especially  if  its  design  is  physics-based.  The  four  models  in  Section  6.1.1  remain  applicable  to  help¬ 
ing  characterize  the  uncertainty  in  the  state  of  an  object.  The  use  of  process  noise  such  as  these  four 
models  does  not  replace  the  goal  of  developing  good  physical  models,  but  should  be  viewed  as  an 
approach  to  account  for  the  uncertainties  in  the  models. 

4.  Sections  6.6  and  6.7  present  batch  formulations  of  the  continuous  and  discrete  problems,  respectively. 
Of  particular  note  is  that  these  can  be  treated  in  a  controls  formulation  in  which  the  noise  is  viewed 
as  a  control  that  is  determined  as  part  of  the  solution  (state  and  control).  This  approach  as  noted  in 
Section  3.3  presents  a  method  for  treating  structural  uncertainty.  A  comparison  of  the  controls  and 
noise  approaches  should  be  investigated  and  compared. 

5.  The  use  of  empirical  covariances  described  in  the  work  of  Frisbee  [155]  and  Cerven  [47,  48]  and 
briefly  summarized  in  Section  6.8  appears  to  be  promising  and  should  be  further  studied  to  see  if  it 
could  lead  to  improvements  in  the  representation  of  uncertainty  in  the  state  of  a  space  object. 

6.  While  batch  estimation  formulations  are  quite  classical  (e.g.,  Jazwinski  [139]  and  Escobal  [86]),  the 
development  of  robust  numerical  methods  is  more  recent.  A  recommendation  is  to  make  use  of  a 
globalization  technique  such  as  a  line  search  or  trust  region  method  to  improve  the  convergence  of 
the  methods  to  solutions  whose  errors  are  substantially  below  those  in  the  measurements  and  as- 
trodynamic  models.  Also,  one  finds  that  many  of  the  nonlinear  least  squares  problems  arising  in 
astrodynamics  often  have  multiple  optima,  so  some  care  is  needed  in  selecting  the  correct  solution. 

7.  The  measurements  from  a  short  arc  of  sensor  reports  are  frequently,  if  not  most  often,  correlated 
while  the  above  estimation  techniques  assume  they  are  uncorrelated.  Thus,  attention  to  the  correlation 
is  needed  to  improve  performance  and  achieve  covariance  realism  during  orbit  determination. 

8.  Algorithms  that  are  capable  of  representing  the  entire  PDF  or  at  least  higher  moments  should  be 
considered  for  propagating  uncertainty. 
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Chapter  7 


Propagation  of  Uncertainty 


7.1  Introduction  to  the  Propagation  of  Uncertainty 

The  propagation  of  uncertainty  from  epoch,  i.e.,  at  the  time  of  the  last  measurement  update,  is  required  to 
support  the  mission  areas  of  conjunction  analysis,  data  association,  anomaly  detection,  and  sensor  tasking 
and  scheduling.  These  uncertainties  fall  into  two  broad  classes,  namely  aleatoric  and  epistemic  uncertainties. 
The  former  are  generally  treated  within  a  probabilistic  framework  and  the  latter,  by  other  methods  such  as 
fuzzy  logic  and  evidence  theory  (Dempster-Shafer  theory)  [253, 258].  The  methods  addressed  in  this  chapter 
are  of  the  aleatoric  type.  What  is  generally  assumed  is  that  the  uncertainty  is  represented  by  a  Gaussian 
random  variable  at  epoch;  however,  this  is  not  necessary.  Regardless  of  the  representation,  the  probability 
density  function  (PDF)  must  be  propagated  through  the  nonlinear  dynamical  system  (6.1.2). 

In  addition  to  the  methods  for  propagating  uncertainty,  there  are  several  considerations  that  affect  the 
quality  of  the  representation  of  uncertainty  and  the  length  of  time  it  remains  valid.  First,  the  coordinate 
system  in  which  uncertainty  is  represented  and  propagated  has  a  significant  impact.  Generally  this  means 
a  Cartesian  versus  an  orbital  element  space  as  discussed  in  Section  7.2.  Regardless  of  which  coordinate 
system  is  used,  each  of  the  aforementioned  mission  areas  may  require  the  representation  in  a  different  co¬ 
ordinate  system  than  that  of  the  propagated  uncertainty.  Thus,  one  must  address  the  faithful  representation 
in  the  needed  coordinate  system.  Other  variables  in  testing  the  different  approaches  include  the  following: 
a)  inclusion/exclusion  of  model  mismatch  (i.e.,  the  difference  between  the  ’’truth”  model  and  the  hypoth¬ 
esized  model),  b)  inclusion/exclusion  of  process  noise  or  stochastic  acceleration,  c)  the  size  of  the  initial 
uncertainty,  e.g.,  the  size  of  the  semi-major  axis  variance,  and  d)  computational  complexity. 

Research  into  the  topic  of  “propagation  of  uncertainty”  by  the  astrodynamics  community  is  broad  and 
quite  excellent.  Thus,  the  goal  of  this  chapter  is  to  survey  most  of  these  methods  and  to  make  recommenda¬ 
tions  on  approaches  that  might  be  used  to  evaluate  the  different  methods  for  different  purposes.  No  doubt, 
different  methods  may  be  required  for  different  applications.  The  framework  for  the  astrodynamic  equations 
of  motion  is  presented  in  Section  4. 1  and  Chapter  6  and  will  not  be  repeated  here. 
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7.2  Coordinate  Systems  for  Representing  Uncertainty  in  SSA 

Under  Gaussian  assumptions,  the  coordinates  used  to  represent  the  state  space  can  impact  how  long  one 
can  propagate  the  uncertainty  under  a  non-linear  dynamical  system;  a  Gaussian  random  vector  will  not  be 
mapped  to  a  Gaussian  under  a  non-linear  transformation,  but  can  remain  nearly  Gaussian  for  some  time 
depending  on  the  coordinate  system.  The  representation  of  a  space  object’s  kinematic  state  in  orbital  ele¬ 
ment  coordinates  [207],  rather  than  Cartesian  Earth-Centered-Inertial  (ECI)  position- velocity  coordinates, 
is  well-suited  to  the  space  surveillance  tracking  problem  since  such  coordinates  “absorb”  the  most  dominant 
terms  in  the  non-linear  gravitational  potential  leading  to  “less  non-linear”  propagations.  Thus,  these  special 
coordinates  can  mitigate  the  departure  from  “Gaussianity”  under  the  non-linear  propagation  of  an  initial 
Gaussian  state  PDF  with  respect  to  orbital  elements  [231]. 

For  example,  the  Keplerian  orbital  elements  (a,  e,  i,  0,  uj,  M )  define  a  system  of  curvilinear  coordinates 
with  respect  to  six-dimensional  position- velocity  space  and  encompass  the  geometric  and  physical  properties 
of  unperturbed  two-body  dynamics.  As  noted  above,  by  absorbing  the  most  dominant  non-linear  term  in  the 
equations  of  motion  (i.e.,  the  1/r2  term),  propagation  is  less  non-linear  in  orbital  elements,  thereby  more 
closely  preserving  any  Gaussian  and  linear  approximations.  Such  orbital  elements  are,  however,  singular  at 
zero  eccentricity  and  inclination  [207].  Thus,  one  turns  to  the  nonsingular  equinoctial  elements,  which  are 
in  wide  use  for  propagating  uncertainty.  Based  on  the  merits  of  the  equinoctial  elements  for  representing 
the  PDF  of  a  space  object’s  orbital  state,  one  can  ask  if  further  improvements  are  possible  by  incorporating 
higher-order  gravity  perturbations  beyond  the  1/r2  contribution.  Indeed,  following  the  work  of  Brouwer 
[42]  and  Fyddane  [183],  one  can  define  a  new  system  of  orbital  elements,  called  the  J2  equinoctial  orbital 
elements,  that  effectively  absorb  the  J2  zonal  correction  (i.e.,  the  most  dominant  term  in  the  geopotential 
expansion  that  accounts  for  the  Earth’s  oblateness)  [10].  Eike  the  traditional  equinoctial  elements  discussed 
above,  the  J2  equinoctial  elements  also  possess  the  desirable  property  of  being  non-singular  for  orbits  with 
zero  eccentricity  or  zero  inclination  [14].  The  use  of  these  different  orbital  element  spaces  to  represent 
uncertainty  and  then  the  transformation  of  the  uncertainty  to  a  Cartesian  space  to  support,  for  example, 
conjunction  assessments  is  discussed  in  Section  7.9. 

7.3  The  Fokker-Planck-Kolmogorov  Equation  for  Continuous  Dynamics 

For  continuous  time  dynamics,  the  dynamical  model  of  the  form  (6. 1 .2)  governs  the  astrodynamics.  Given  an 
initial  probability  density  function  describing  the  uncertainty  in  the  state  of  an  object,  say,  p  =  p(x,  1 1  Zf,_  | 
the  pdf  at  a  future  time  p  =  p(x,  t\Zk~i)  for  t  >  tk-i  is  governed  by  the  Fokker-Planck-Kolmogorov  equa¬ 
tion  (FPKE)  [139] 

%  =  -V£(p/)  +  ^tr[V*V£(pGQGT)],  (7.3.1) 

where  Vx  is  the  gradient  with  respect  to  x  viewed  as  a  column  operator.  In  addition  FPKE,  there  is  a 
corresponding  backward  equation  [139,  p.  130]  that  can  be  used  for  smoothing.  Additional  details  are  given 
in  Section  6.3. 

Analytical  solutions  to  the  FPKE  are  generally  intractable  and  are  only  known  in  a  few  restrictive  cases. 
A  sparse  grid  method  [161]  is  one  possible  approach  to  solving  the  FPKE  numerically;  otherwise,  one  turns 
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CHAPTER  7.  PROPAGATION  OF  UNCERTAINTY 


to  approximate  methods  discussed  in  subsequent  sections. 

7.4  Extended  Kalman  Filter 

General  nonlinear  state  estimation  problems  are  generally  not  solved  exactly.  The  most  widely  used  approxi¬ 
mate  solution  is  the  extended  Kalman  filter  (EKF),  which  applies  the  standard  linear  Kalman  filter  algorithm 
with  linearization  of  the  nonlinear  system.  This  algorithm  requires  that  the  process  (stochastic  acceleration), 
observation  noises,  and  the  initial  random  variable  x(to)  be  independent  white  Gaussian  processes.  In  this 
setting,  given  a  Gaussian,  say  iV(/i o,  Po)>  the  mean  //q  and  covariance  Po  are  propagated  via  the  noiseless 
dynamics  and  Ricatti  equation  dynamics 

=  f(x(t),a,t),  x(t0)  =  Ho  (7.4.1) 

=  F ex,  t) P  +  PTFT(a;(f),  a,  t )  +  G(*(t),  t)Q(t)GT t),  P(t0)  =  P0)  (7.4.2) 

respectively,  where  F(x(t),ot,  t )  =  Dxf(x(t),  a,  t ).  In  case  Q (t)  =  0,  i.e.,  the  noiseless  case, 

P  (t)  =  &(t)P0$(t)T  (7.4.3) 

where 

=  F (x(t),a,  *(t0)  =  I.  (7.4.4) 

A  summary  of  the  EKF  can  be  found  in  the  book  edited  by  Gelb  [96],  the  book  by  Jazwinski  [139],  or  in 
almost  any  book  on  Kalman  filtering. 

Advantages  and  disadvantages  of  the  extended  Kalman  filter  are  discussed  extensively  in  the  vast  litera¬ 
ture.  Vallado  [262,  pp  799-800]  specifically  discusses  these  for  astrodynamics,  so  a  review  of  this  literature 
will  not  be  undertaken  here.  Here  are  some  brief  comments.  A  Gaussian  random  variable  is  propagated 
analytically  through  the  first-order  linearization  of  a  nonlinear  system.  As  such,  the  EKF  can  be  viewed  as 
providing  “first-order”  approximations  to  the  optimal  probability  density  function.  These  approximations, 
however,  can  introduce  large  errors  in  the  true  posterior  mean  and  covariance  of  the  transformed  (Gaussian) 
random  variable,  which  may  lead  to  sub-optimal  performance  and  sometimes  divergence  of  the  filter.  When 
propagation  times  are  longer  or  the  initial  uncertainties  are  large,  the  propagated  covariances  become  unre¬ 
alistic  sooner  than  other  methods  to  follow  and  cannot  capture  the  higher  order  moments.  Also,  covariances 
become  overly  optimistic  when  process  noise  is  omitted.  Proper  accounting  of  dynamical  model  deficien¬ 
cies  through  the  use  of  stochastic  model  parameters  and/or  process  noise  is  required  to  maintain  a  realistic 
covariance. 

7.5  Second  Order  EKFs 

The  second  order  extended  Kalman  filter  (SOEKF)  [139]  is  based  on  a  second  order  Taylor  expansion  of 
a  nonlinear  system,  in  contrast  to  the  more  common  (first  order)  extended  Kalman  filter  (EKF).  Despite 
a  solid  theoretical  ground  for  its  approximation,  it  is  seldom  used  in  applications,  where  the  EKF  and 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


73 


7.6.  COVARIANCE  PROPAGATION  USING  THE  UNSiPEERED  KlXmiGAIMmW  UNCERTAINTY 


the  unscented  Kalman  filter  (UKF)  are  the  standard  algorithms.  One  reason  might  be  the  requirement  for 
analytical  Jacobian  and  Hessian  of  the  system  equations,  e.g.,  the  gravity  model,  and  the  high  complexity 
that  scales  with  the  state  order.  An  exception  is  the  work  of  Park  and  Scheeres  [218]  and  Vittaldev,  Russell, 
Arora,  and  Gaylor  [274]  in  astrodynamics. 

7.6  Covariance  Propagation  using  the  Unscented  Kalman  Filter 

The  unscented  Kalman  filter  (UKF)  uses  the  unscented  transform  for  both  the  discrete  state  and  measure¬ 
ment  equations,  i.e..  Equations  (6.1.3)  and  (6.1.4),  respectively.  It  provides  a  minimum-mean-squared  error 
(MMSE)  estimate  of  the  state.  The  UKF  was  first  introduced  by  Julier,  Uhlmann,  and  Durant- Whyte  [157]  in 
2000  and  can  also  be  viewed  as  a  central  difference  filter  [212].  A  novel  feature  of  the  UKF  is  its  “derivative- 
free”  nature.  (Such  a  feature  mitigates  some  of  the  complexity  in  trying  to  take  higher  order  derivatives  of 
the  gravity  model.)  Within  the  filter  prediction  step,  the  propagated  state  estimate  and  covariance  are  re¬ 
constructed  from  a  deterministically  chosen  set  of  “sigma  points”  propagated  through  the  full  non-linear 
dynamics.  The  equivalence  between  this  reconstruction  or  “unscented  transform”  and  Gauss-Hermite  (GH) 
quadrature  has  been  established  [125,  138],  Variations  and  extensions  of  the  UKF,  including  a  more  numer¬ 
ically  stable  “square -root  version”  and  the  higher-order  GH  filters,  have  also  been  formulated  [138,  269].  A 
detailed  description  of  the  algorithm  for  the  UKF  as  it  applies  to  the  discrete  system  in  Equations  (6.1.3)  and 
(6.1.4),  including  process  noise,  can  be  found  in  the  paper  by  Wan  and  Merwe  [281]  and  in  Section  3.2.3 
of  the  thesis  by  Merwe  [268].  A  more  recent  description  is  that  of  Stauch  and  Jah  [249]  who  combine  the 
unscented  Kalman  filter  with  the  Schmidt-Kalman  filter  into  a  unscented  Schmidt-Kalman  filter. 

At  a  high  level,  covariance  propagation  using  the  unscented  transform  (or,  equivalently,  Gauss-Hermite 
quadrature  [138])  works  as  follows.  The  initial  uncertainty  is  assumed  to  be  Gaussian  with  respect  to 
some  state  coordinates,  for  example,  Cartesian  ECI  position-velocity  coordinates  or  (equinoctial)  orbital 
elements.  For  the  former,  one  first  deterministically  selects  a  sequence  of  sigma  points  (i.e.,  Gauss-Hermite 
quadrature  nodes)  and  weights  from  the  initial  Gaussian  in  ECI  space.  Each  of  these  sigma  points  is  then 
propagated  to  a  future  epoch.  Fastly,  the  state  and  covariance  at  the  final  epoch  are  reconstructed  using 
the  weights  and  propagated  sigma  points.  For  the  case  in  which  the  initial  Gaussian  is  represented  in  a 
space  other  than  Cartesian  ECI  coordinates  (say  equinoctial  elements  for  sake  of  example),  one  proceeds 
similarly  by  first  selecting  the  sigma  points  and  weights  in  equinoctial  space.  Each  sigma  point  is  then 
transformed  to  Cartesian  space  and  then  propagated  to  a  future  epoch*  (as  in  the  previous  case).  Finally, 
each  propagated  sigma  point  is  transformed  back  to  the  equinoctial  space  and  the  Gaussian  at  the  final 
epoch  is  recovered  using  the  weights  and  propagated  (equinoctial)  sigma  points.  The  transformation  of 
covariance  and  mean  back  to  orbital  elements  may  require  Gaussian  sums  as  discussed  in  the  paper  [16]. 
Some  additional  comments  on  covariance  propagation  using  the  UKF  are  provided  below. 

1.  The  unscented  transform  used  in  the  UKF  is  equivalent  to  a  third-order  GH  quadrature  rule  [138]. 
Such  a  third-order  rule  can  be  obtained  with  2n  +  1  sigma  points  (nodes)  where  n  is  the  dimension  of 
the  state  space. 

*The  reason  for  this  transformation  is  that  the  dynamics  modeled  by  the  ODEs  in  (6.1.2)  are  commonly  defined  in  ECI  space. 
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2.  In  practice,  negligible  differences  arc  observed  between  the  propagated  state  and  covariance  obtained 
using  the  traditional  unscented  transform  (i.e.,  a  third-order  GH  rule)  and  a  higher-order  GH  rule 
[121].  This  suggests  that  the  standard  UKF  resolves  the  correct  mean  and  covariance  of  the  propagated 
uncertainty;  i.e.,  it  achieves  covariance  realism. 

3.  One  can  choose  any  numerical  method  for  propagating  the  sigma  points  such  as  explicit  or  implicit 
Runge-Kutta,  Adams-Bashforth-Moulton,  or  Gauss-Jackson.  No  matter  what  propagator  is  used,  the 
sigma  points  can  be  propagated  in  parallel.  A  finer  level  of  parallelization  is  also  possible  using 
implicit  Runge-Kutta  methods,  wherein  each  force-model  evaluation  can  be  parallelized.  Implicit 
Runge-Kutta  are  also  well-suited  for  sigma-point  propagations  since  recent  developments  [12,  14] 
have  demonstrated  that  the  sigma  points  can  be  propagated  collectively,  rather  than  individually, 
leading  to  improved  computational  efficiency  even  in  a  serial  computing  environment.  Further,  as 
remarked  in  the  comments  on  the  EKF,  the  numerical  truncation  error  committed  by  the  orbital  prop¬ 
agator  should  be  significantly  below  the  uncertainty  in  the  model  or  physics  [15]. 

4.  If  continuous-time  process  noise  is  modeled  in  the  dynamics  through  a  stochastic  differential  equation, 
one  can  instead  numerically  integrate  an  analogous  Riccati  equation  for  the  covariance  following  the 
work  of  Sarkka  [232].  Alternatively,  one  can  replace  the  underlying  stochastic  differential  equation  by 
an  approximate  discrete-time  formulation  of  the  dynamics  (with  discrete-time  process  noise)  thereby 
allowing  one  to  use  the  unscented  transform  as  described  above  in  conjunction  with  a  standard  orbital 
propagator. 

5.  If  observable,  uncertain  parameters  in  the  dynamics  can  be  treated  using  state  augmentation  or  via  an 
extension  of  the  Schmidt-Kalman  filter  [139]. 

6.  The  computed  “state  estimate”  from  the  unscented  transform  (GH  quadrature)  is  the  mean  of  the 
propagated  probability  distribution  function,  which  can  be  different  from  the  mode  of  the  true  PDF 
(see,  for  example,  Figure  7.10.1  discussed  in  Subsection  7.10.2). 

Here  is  a  list  of  some  advantages  of  the  UKF. 

•  The  UKF  is  derivative  free,  i.e.,  no  explicit  calculation  of  Jacobians  or  Hessians  are  necessary  to 
implement  the  algorithm. 

•  The  overall  number  of  computations  are  higher  than  that  of  the  EKF;  however,  if  one  propagates 
the  sigma  points  collectively,  the  computations  can  be  reduced  to  about  the  same  as  the  EKF  (see 
Section  7.15  and  [15,  123]). 

•  The  UKF  when  combined  with  an  orbital  element  space  in  which  the  uncertainty  is  represented  can 
considerably  extend  the  life  of  a  faithful  representation  of  the  uncertainty  longer  than  the  EKF  in  a 
Cartesian  space. 

Some  disadvantages  of  the  UKF  are  as  follows. 

•  The  sigma  points  must  be  computed  with  sufficient  accuracy  to  allow  for  a  meaningful  reconstruction 
of  the  mean  and  covariance  of  the  transformed  sigma  points. 
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•  The  UKF  still  breaks  down  as  do  all  of  the  approximate  filters;  however,  Gaussian  sums  and  the  GVM 
filter  in  the  next  sections  can  extend  the  life  of  the  UKF. 

•  The  UKF’s  range  of  validity  is  more  limited  than  that  of  a  particle  filter;  however,  the  computational 
cost  is  substantially  less. 


7.7  Uncertainty  Propagation  using  the  Gauss  von  Mises  Filter 

Because  uncertainty  propagation  using  the  UKF  only  captures  the  first  two  cumulants  (i.e.,  mean,  covari¬ 
ance)  of  the  (possibly  non-Gaussian)  state  PDF,  higher-order  cumulants  (e.g.,  skewness,  kurtosis)  can  be¬ 
come  non-negligible  in  scenarios  involving  propagations  over  extended  time  intervals  in  the  absence  of 
measurement  updates.  In  such  cases,  what  is  needed  to  achieve  uncertainty  realism  is  a  representation  of 
the  state  PDF  beyond  a  simple  Gaussian,  such  as  the  newly  proposed  Gauss  von  Mises  (GVM)  distribution 
[126,  127],  which  is  designed  to  work  in  orbital  element  space  and  not  a  Cartesian  space.  Some  of  the  salient 
features  of  GVM  distribution  are  presented  in  this  section;  the  detailed  explanations  can  be  found  in  [127]. 

The  random  variables^  (x.  9)  E  M”  x  §  are  said  to  be  jointly  distributed  as  a  Gauss  von  Mises  (GVM) 
distribution  if  and  only  if  their  joint  probability  density  function  has  the  form 

p( x,  9)  =  QVM(x,  6;  /i,  P,  a,  (3 ,  T,  k)  =  A f{x\ p,,  P)  VM(9;  @(x),n), 


where 


Af(x;p,P)  =  - 
VM(9;  ©(as),  k)  = 


1 


Y /  det(27rP] 
1 

27re_K/o(^) 


exp 


exp 


-^(*-/*)Tp  1{x~P) 


—2k sin2  -(9  —  ©(*)) 


and 

&(x)  =  a  +  (3tz  +  1 2 1  Fz,  z  =  A_1(ai  —  /x),  P  =  AAJ . 

The  parameter  set  (p,  P,  a,  [3,  T,  n)  is  subject  to  the  following  constraints:  p,  E  Mr\  P  is  an  n  x  n  symmetric 
positive-definite  matrix,  a  E  M,  (3  E  Mn,  T  is  an  n  x  n  symmetric  matrix,  and  k  A  0.  The  matrix  A  in  the 
definition  of  the  normalized  variable  z  is  the  lower-triangular  Cholesky  factor  [102]  of  the  parameter  matrix 
P.  Some  key  properties  of  the  GVM  distribution  are  listed  below. 


1 .  The  GVM  distribution  is  a  PDF  defined  on  the  cylindrical  manifold  MTl  x  §  that  satisfies  the  periodicity 
property  p(x,  9)  =  p(x.  9  +  2irk),  for  any  integer  k.  The  GVM  filter  treats  the  variable  9  as  a  bona 
fide  angular  coordinate,  within  the  framework  of  direction  statistics  [192],  rather  than  an  unbounded 
Cartesian  coordinate.  Space  object  orbital  states  in  (traditional  or  ,/o  [10])  equinoctial  orbital  elements 
can  be  represented  rigorously  using  the  GVM  distribution  in  which  the  variable  x  is  identified  with 
the  first  five  elements  (a,  h,  k,p ,  q)  and  the  angular  variable  9  is  identified  with  the  mean  longitude 
coordinate  I. 

^The  notation  R™  denotes  n-dimensional  (flat)  Euclidean  space.  The  notation  S  denotes  the  one-dimensional  manifold  { ( x ,  y)  £ 
R2  |  x2  +  y2  =  1};  i.e.,  the  circle.  Thus,  Rn  x  S  denotes  the  (n  +  1) -dimensional  cylindrical  manifold. 
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2.  The  GVM  distribution  is  unimodal  on  x  §  with 


Qu,  a)  =  argmax^g  QVM(x,  0;  p,  P,  a,  (3..  T,  k). 

3.  The  parameters  (3  and  T  in  the  GVM  distribution  model  correlation  between  x  and  6.  In  particular,  the 
parameter  matrix  T  can  be  tuned  to  give  the  level  sets  of  the  GVM  distribution  then-  distinctive  banana 
or  boomerang  shape  (see,  for  example,  Figure  7.10.1),  thereby  allowing  one  to  model  higher-order 
cumulants. 

4.  The  GVM  distribution  reduces  to  a  Gaussian  distribution  in  the  limit  as  T  — >  0  and  n  h>  oc. 

5.  It  follows  that  [127]  the  scalar 

U(x,6]  p,P,a,  /3,T,  n)  =  (x  —  p)TP~1(x  —  p)  +  4k  sin2  ^(0  —  O(x)),  (7.7.1) 

possesses  the  distribution  x2(n  +  1)  in  the  limit  as  n  — »•  oc.  This  is  precisely  the  uncertainty  realism 
metric  proposed  in  the  companion  paper  [124]  that  generalizes  the  Mahalanobis  distance  of  a  Gaussian 
random  vector  x  with  mean  p  and  covariance  P. 

We  now  give  a  high-level  description  of  uncertainty  propagation  using  the  GVM  filter  will  full  details 
provided  in  Horwood  and  Poore  [127].  The  initial  uncertainty  is  assumed  to  be  a  GVM  distribution  with 
respect  to  some  system  of  orbital  element  coordinates  (e.g.,  equinoctial).  If  the  initial  distribution  is  a  Gaus¬ 
sian,  it  can  be  converted  to  a  GVM  distribution  as  described  in  reference  [127].  By  analogy  to  the  unscented 
transform,  one  begins  by  deterministically  selecting  a  sequence  of  sigma  points  and  weights  from  the  ini¬ 
tial  GVM  distribution.  Note  that  the  prescription  for  choosing  such  sigma  points  and  weights  differs  from 
that  used  in  the  unscented  transform  or  Gauss-Hermite  quadrature.  Each  sigma  point  is  then  transformed 
to  Cartesian  space  and  then  propagated  to  a  future  epoch  (with  dynamics  governed,  for  example,  by  the 
ODEs  (6.1.2)).  Finally,  each  propagated  sigma  point  is  transformed  back  to  the  underlying  orbital  element 
space  and  the  parameter  set  (p,  P,  a ,  (3,  T,  k)  of  the  GVM  distribution  at  the  final  epoch  is  recovered  us¬ 
ing  the  weights  and  propagated  (orbital  element)  sigma  points.  Some  additional  comments  on  uncertainty 
propagation  using  the  GVM  distribution  are  provided  below. 

1.  Uncertainty  propagation  using  the  (third-order)  GVM  filter  only  requires  the  propagation  of  13  sigma 
points  (for  a  6D  equinoctial  orbital  element  state);  hence  it  has  approximately  the  same  computational 
cost  as  the  UKF.  Higher-order  versions  of  the  GVM  filter  are  also  possible.  Higher  order  versions  of 
the  GVM  filter  are  possible  such  as  the  third-order  version  of  the  GVM  filter  and  a  fifth-order  GVM 
filter  with  the  latter  requiring  73  sigma  points  [124]. 

2.  In  practice,  we  observe  that  the  GVM  filter  effectively  “extends  the  life  of  the  UKF,”  by  maintaining 
uncertainty  realism  longer  than  the  UKF,  up  to  7  times  in  many  examples.  Evidence  is  provided  in 
references  [126,  127]. 

3.  Fike  the  UKF,  the  GVM  filter  is  compatible  with  standard  orbital  propagators  as  well  as  implicit 
Runge-Kutta  methods  that  exploit  collective  propagation  of  the  sigma  points  for  improved  computa¬ 
tional  efficiency  [12,  14]. 
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In  summary,  the  GVM  provides  improved  covariance  and  uncertainty  realism  at  no  additional  computa¬ 


tional  cost  compared  to  traditional  methods  for  uncertainty  propagation.  In  particular,  some  advantages  of 


the  GVM  filter  are  as  follows: 

1 .  Operates  on  a  new  family  of  multivariate  PDFs  that  provide  a  statistically  rigorous  treatment  of  un¬ 
certainty  in  orbital  element  space. 

2.  Provides  both  covariance  and  uncertainty  realism  by  modeling  higher-order  cumulants  (e.g.,  skewness 
and  kurtosis)  beyond  a  state  and  covariance. 

3.  Reduces  to  a  Gaussian  for  a  subset  of  the  parameter  space. 

4.  Has  the  same  computational  cost  as  the  traditional  UKF  (i.e.,  only  13  “sigma  points”  or  particles  need 
to  be  propagated  for  uncertainty  propagation  of  a  6D  orbital  state). 

5.  Admits  a  natural  extension  of  the  classical  Mahalanobis  distance  that  is  useful  in  estimation  and 
tracking  applications. 

6.  Can  be  extended  to  a  mixture  filter,  to  provide  improved  accuracy  in  extreme  cases  at  over  a  95 
reduced  cost  compared  to  Gaussian  mixtures. 

Disadvantages  of  the  GVM  filter  are  similar  to  those  of  the  UKF.  Other  disadvantages  are  as  follows. 

•  The  sigma  points  must  be  computed  with  sufficient  accuracy  to  allow  for  a  meaningful  reconstruction 
of  the  mean  and  covariance  of  the  transformed  sigma  points. 

•  The  GVM  still  breaks  down  as  do  all  of  the  approximate  filters;  however,  mixture  versions  of  the 
GVM  filter  can  extend  the  life  of  the  GVM. 

•  The  GVM’s  range  of  validity  is  more  limited  than  that  of  a  particle  filter;  however,  the  computational 
cost  is  substantially  less. 

•  The  GVM  represents  uncertainty  in  an  element  space.  Thus,  in  applications  requiring  a  Cartesian 
representation  of  the  uncertainty,  the  GVM  distribution  (in  element  space)  must  be  transformed  to  a 
Cartesian  space  using,  for  example,  a  Gaussian  sum  as  described  in  Section  7.9. 


7.8  Gaussian  Sum  Filters 


A  Gaussian  sum  is  a  mixture  density  of  the  form 


N 


where  the  weights  wa  are  non-negative  scalars  which  sum  to  unity  and  N(x\  fi,  P)  denotes  the  Gaussian 
PDF  with  mean  fi  and  covariance  P;  i.e., 


-\/det(27rP)  L  2 


(7.8.1) 
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The  Gaussian  sum  filter  (GSF)  is  based  on  a  fundamental  result  of  Alspach  and  Sorenson  [7]  which  states 
that  any  PDF  can  be  approximated  arbitrarily  close  (in  the  L1  sense)  by  a  weighted  sum  (mixture)  of  Gaus¬ 
sian  PDFs  henceforth  called  a  Gaussian  sum.  Thus,  Gaussian  sums  provide  a  mechanism  for  modeling 
non-Gaussian  densities  and  for  more  accurately  approximating  the  solution  of  the  FPKE.  Computationally, 
the  GSF  has  the  added  advantage  of  being  parallelizable  since  filters  such  as  the  EKF  or  UKF  act  inde¬ 
pendently  on  each  component  Gaussian  in  the  prediction  and  correction  steps.  With  regards  to  the  weights, 
means,  and  covariances  in  the  Gaussian  sum  approximation,  there  are  two  key  problems.  Th e first  is  to  select 
these  via  the  L 1  theory  to  represent  the  given  probability  density  function.  The  second  is  to  adapt  these  as 
the  true  probability  density  function  changes  due  to  its  being  propagated  forward  in  time  or  is  transformed 
through  a  coordinate  system  change,  e.g.,  equinoctial  orbital  elements  to  a  Cartesian  system. 

A  method  for  the  first  problem  is  discussed  in  in  the  paper  by  Florwood,  Aragon,  and  Poore  [122]  in 
which  a  covariance  or  Gaussian  sum  is  represented  in  equinoctial  orbital  element  space  with  component 
means,  covariances,  and  weights  initially  selected  (by  solving  an  L2  optimization  problem  offline)  such 
that  the  (square-root  version  of  the)  UKF  [269],  when  acting  in  parallel  on  each  component,  accurately 
approximates  the  solution  of  the  Fokker-Planck-Kolmogorov  equation  (FPKE).  The  number  of  Gaussian 
components  required  to  achieve  an  accurate  approximation  is  chosen  adaptively  based  on  the  length  of  the 
propagation  time  and  the  initial  error  (standard  deviation)  along  the  radial  direction  (semi-major  axis  coor¬ 
dinate).  Consequently,  by  representing  the  PDF  in  the  equinoctial  elements,  the  algorithm  achieves  superior 
computational  efficiency  because  it  only  requires  a  Gaussian  sum  along  one  dimension.  The  refinement 
methodology  is  illustrated  in  Figure  7.8.1.  Under  a  nonlinear  transformation,  a  Gaussian  (represented  by  the 
thick  black  ellipse)  need  not  be  mapped  to  a  Gaussian  (e.g.,  the  level  surfaces  of  the  transformed  distribution 
could  look  crescent-shaped). 


Figure  7.8.1  Depiction  of  a  single  Gaussian  and  its  Gaussian  sum  approximation  undergoing  a  nonlinear 
transformation. 


However,  in  a  sufficiently  small  neighborhood,  any  (smooth)  nonlinear  map  will  be  approximately  lin¬ 
ear.  Consequently,  Gaussians  with  smaller  covariances  (represented  by  the  colored  elliptic  disks)  remain 
more  Gaussian  than  those  with  larger  covariances  under  the  nonlinear  mapping.  Therefore,  a  Gaussian 
sum  refined  by  approximating  each  constituent  Gaussian  by  a  finer  Gaussian  sum  will  exhibit  better  be¬ 
havior  through  nonlinear  transformations.  It  suffices  to  optimally  refine  the  unit  one-dimensional  Gaussian 
J\T(x,0, 1);  refinement  of  a  multivariate  Gaussian  with  an  arbitrary  covariance  is  obtained  by  a  series  of 
linear  transformations  detailed  in  the  paper  by  Horwood,  Aragon,  and  Poore  [122].  A  metric  for  ranking 
directions  according  to  their  nonlinearity  and  only  splitting  along  the  necessary  directions  can  be  found  in 
the  work  of  Vittaldev  and  Russell  [273]. 

For  the  second  problem,  several  have  proposed  methods  for  adapting  the  weights  based  on  various  online 
L2  optimization  criteria  [62,  100,  255,  271,  272].  As  we  note,  the  weights  are  chosen  or  adapted  using  an 
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L 2  optimization  criteria  which  is  different  from  the  basic  theory  in  L1.  A  method  for  this  is  to  solve  the 
L2  optimization  problem  and  then  verify  the  accuracy  in  L1  as  has  been  done  in  the  paper  by  Horwood, 
Aragon,  and  Poore  [122]. 

Some  advantages  of  the  Gaussian  sum  representation  of  uncertainty  include. 

•  The  weights,  means,  and  covariances  in  a  Gaussian  sum  PDF  can  represent  an  arbitrary  PDF  in  the 
L 1  sense  according  to  the  theory  of  Alspach  and  Sorenson  [7]  . 

•  With  either  the  EKF  or  UKF,  the  propagation  is  parallelizable. 

•  A  brute  force  method  such  as  Monte  Carlo  methods  using  massive  parallelization  on  GPUs  may  make 
the  computations  tractable  in  some  cases  [20]. 

Disadvantages  of  the  Gaussian  sum  filters  include  the  following. 

•  The  optimization  problem  that  one  often  solves  is  that  in  L2  instead  of  L 1  . 

•  The  computational  complexity  of  a  Gaussian  sum  filter  generally  indicates  that  they  are  for  special 
puiposes  such  as  propagating  a  Gaussian  over  a  long  time  period. 


7.9  Transformation  of  Uncertainty  from  an  Element  Space  to  ECI 

The  paper  by  Aristoff,  Horwood,  Singh,  and  Poore  [16]  demonstrates  how  uncertainty  in  orbital  elements 
can  be  faithfully  (and  efficiently)  transformed  to  Cartesian  space  using  Gaussian  mixtures.  Thus,  this  paper 
brings  to  a  completion  the  initial  representation  in  an  element  space,  the  propagation  forward  in  time,  the 
representation  again  in  element  space,  and  the  final  transformation  back  to  a  Cartesian  space.  For  example, 
given  the  Gaussian  sum  representation  in  a  Cartesian  space,  one  can  use  the  representation  in  the  work  of 
DeMars,  Cheng,  and  Jah  [65]  or  Vittaldev  and  Russell  [272,  273]  to  compute  the  probability  of  collision. 

In  other  work,  Weisman  et  al.  [282-284]  demonstrate  probability  density  function  mapping  between 
measurement  space  and  state-spaces  of  interest  (Cartesian,  Keplerian,  Gim-Alfriend,  and  mean  Keplerian) 
as  well  as  the  impact  of  initial  orbit  determination  processes  on  the  state  distribution  utilizing  the  transfor¬ 
mation  of  variables  [217].  This  mapping  produces  exact  knowledge  of  the  system  likelihood  distribution 
to  allow  for  a  better  idea  of  the  combination  of  system  states  which  generated  the  measurement.  Since  the 
likelihood  distribution  is  exactly  mapped  between  domains,  Bayesian  estimation  can  easily  be  carried  out 
if  the  prior  distribution  is  appropriately  characterized.  The  posterior  probability  density  function  computed 
by  Bayes’  Theorem  allows  for  all  statistical  moments  to  be  assessed,  not  just  the  mean  and  covariance  as 
with  conventional  filtering  techniques.  Availability  of  the  state  probability  density  function  given  a  single 
measurement  set  can  allow  for  automation  of  the  covariance  initialization  required  by  conventional  filtering, 
thereby  decreasing  the  amount  of  tuning  needed  to  ensure  proper  filter  operation.  The  method  requires  an 
analytic  form  of  the  starting  probability  density  function  to  be  given  as  well  as  the  mapping  equations  be¬ 
tween  the  starting  space  and  the  desired  space  of  interest,  additionally  the  number  of  states  in  the  desired 
space  must  be  equal  to  or  less  than  the  number  of  states  in  the  starting  space.  With  regard  to  propagation  of 
probability  density  functions,  Majji,  Weisman,  and  Alfriend  [189]  were  able  to  demonstrate  that  if  an  initial 
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probability  density  function  was  analytically  available  then  the  propagation  of  uncertainty  for  Two-Body 
motion  in  Keplerian  variables  without  any  perturbations  was  able  to  be  effectively  carried  out  through  use  of 
the  technique.  It  was  shown  that  the  solution  flow  of  the  Keplerian  variables  inherently  satisfied  Liouville’s 
Equation,  the  Fokker-Planck-Kolmogorov  equation  without  diffusion,  due  to  the  linear  time  update  for  the 
mean  anomaly. 

7.10  Some  Computational  Comparisons 

In  this  section,  an  example  is  presented  to  demonstrate  the  performance  of  some  of  the  above  approaches 
to  propagating  uncertainty.  The  example  is  a  scenario  in  low  Earth  orbit  (LEO)  which  compares  the  GVM 
filter  prediction  step  with  that  of  the  extended  Kalman  filter  (EKF)  [139],  the  unscented  Kalman  filter  (UKF) 
[157],  a  Gaussian  sum  filter  (GSF)  [122],  and  a  particle  filter  [228].  The  accuracy  of  the  GVM  uncertainty 
propagation  algorithm  is  also  validated  using  a  metric  based  on  the  L2  error.  For  the  specific  LEO  scenario,  it 
is  shown  that  the  GVM  filter  prediction  step  properly  characterizes  the  actual  uncertainty  of  a  space  object’s 
orbital  state  while  simple  less  sophisticated  methods  which  make  Gaussian  assumptions  (such  as  the  EKF 
and  UKF)  do  not.  Specifically,  under  the  non-linear  propagation  of  two-body  dynamics,  the  new  algorithm 
properly  characterizes  the  uncertainty  for  up  to  eight  times  as  long  as  the  standard  UKF  all  at  no  additional 
computational  cost  to  the  UKF.  In  what  follows,  the  particulars  of  the  simulation  scenario  are  defined  in 
Subsection  7.10.1  and  the  results  are  discussed  in  Subsection  7.10.2. 

7.10.1  Scenario  Description 

This  subsection  describes  the  specific  input  to  the  uncertainty  propagation  algorithm  in  Sections  7.4-7. 8, 
how  the  output  is  visualized,  and  how  the  accuracy  of  the  output  is  validated. 

7.10.1.1  Input 

The  initial  GVM  distribution  of  the  space  object’s  orbital  state  is  defined  with  respect  to  equinoctial  orbital 
elements  (a,  h,  k,p,  q,  £)  G  M5  x  §  with  parameter  set  (/z,  P,  a,  (3,  T,  k)  given  by 

/z  =  (7136.635  km,  0, 0, 0, 0)T,  P  =  diag((20  km)2, 1(T6, 10-6, 1(T6, lO"6) , 
a  =  0,  (3  =  0 ,  T  =  0,  k  =  3.282806  x  107. 

The  mode  of  this  distribution  describes  a  circular,  non-inclined  orbit  in  LEO  with  a  semi-major  axis  of 
7136.635  km.  This  choice  of  semi-major  axis  is  made  so  that  the  instantaneous  orbital  period  of  the  object 
is  100  minutes.  In  all  subsequent  discussions,  a  time  unit  of  one  orbital  period  is  equal  to  100  minutes.  It 
is  noted  that  the  GVM  distribution  with  the  above  parameter  set  is  approximately  Gaussian  (since  T  =  0 
and  k  1).  In  particular,  the  standard  deviation  in  the  mean  longitude  coordinate  £  is  ac  =  1/yftz  = 
0.01°  =  36".  Finally,  it  is  acknowledged  that  the  initial  parameter  set  defined  above  is  representative  of 
the  uncertainty  of  a  short  radar  track  segment  that  has  yet  to  be  correlated  to  an  existing  object  in  the  space 
catalog.  Due  to  the  sensitivity  of  releasing  any  real  data  acquired  by  the  authors,  this  “sanitized  example”, 
motivated  from  the  real  data,  is  used  instead. 
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The  input  parameters  are  the  epoch  time  to  °f  the  input,  the  final  epoch  time  t,  and  the  specific  forces 
used  to  model  the  perturbations.  In  all  simulations,  to  is  the  J2000  epoch  (1  Jan  2000,  12:00  UTC)  and 
t  —  to  is  varied  from  0.5  to  8  orbital  periods.  The  EGM96  gravity  model  of  degree  and  order  70  is  used  to 
model  the  perturbations.  Though  one  can  include  additional  non-conservative  forces  such  as  atmospheric 
drag,  the  non-linearities  induced  by  the  gravity  alone  (especially  in  LEO)  are  sufficiently  strong  to  stress 
the  algorithm.  Finally,  the  numerical  integration  of  the  ordinary  differential  equations  is  performed  using  a 
Gauss-Jackson  method. 

7.10.1.2  Visualization 

The  output  of  the  uncertainty  propagation  algorithm  in  Section  7.7  is  a  GVM  distribution  characterizing 
the  uncertainty  of  the  space  object’s  orbital  state  at  a  specified  future  epoch.  In  order  to  visualize  this  six¬ 
dimensional  probability  density  function  (PDF),  the  level  curves  (i.e.,  curves  of  equal  likelihood)  of  the 
two-dimensional  (2D)  marginal  PDF  in  the  semi-major  axis  a  and  mean  longitude  I  coordinates  are  plotted. 
In  each  subfigure,  the  a  and  I  coordinates  are  rotated  so  that  the  principal  axes  of  the  EKF  Gaussian  are 
aligned  with  the  horizontal  and  vertical.  (Any  such  rotation  or  rescaling  does  not  exaggerate  any  non- 
Gaussian  effects  or  the  extremity  of  the  boomerang  shape  because  all  cumulants  of  order  three  and  higher 
are  invariant  under  an  affine  transformation.)  This  choice  is  made  because  it  is  along  this  particular  2D  slice 
where  the  greatest  departure  from  “Gaussianity”  and  the  most  extreme  “banana”  or  “boomerang”  shaped 
level  curves  are  observed. 

In  the  panels  of  Figure  7.10.1,  the  na  level  curves  (of  the  marginal  PDF  in  the  a  and  £  coordinates)  are 
plotted  in  half-sigma  increments  starting  at  n  =  '  to  n  =  3  for  various  final  epoch  times  t.  In  order  to 
visualize  these  marginal  PDFs  which  are  defined  on  a  cylinder,  the  cylinder  is  cut  at  a  —  n  (where  a  is  the  a 
parameter  of  the  propagated  GVM  distribution)  and  rolled  out  to  form  a  2D  plane.  This  plane  is  then  rotated 
so  that  the  semi-major  and  semi-minor  axes  of  the  osculating  Gaussian  covariance  are  aligned  with  the 
horizontal  and  vertical  and  then  compressed.  (Any  such  rotation  or  rescaling  does  not  exaggerate  any  non- 
Gaussian  effects  or  the  extremity  of  the  boomerang  shape  because  all  cumulants  of  order  three  and  higher 
are  invariant  under  an  affine  transformation.)  These  level  curves  are  shown  in  shades  of  orange  and  red,  as 
indicated.  Where  appropriate,  overlays  of  the  level  curves  generated  from  the  EKF  and  UKF  are  shown  in 
green  and  grey,  respectively.  Additionally,  blue  crosses  represent  particles  generated  from  a  Monte-Carlo- 
based  uncertainty  propagator.  If  the  represented  PDF  properly  characterizes  the  actual  uncertainty,  then 
approximately  98.9%  of  the  particles  should  be  contained  within  the  respective  3 a  level  curve. 

7.10.1.3  Validation 

An  inspection  of  the  panels  in  Figure  7. 10. 1  provides  a  simple  visual  means  to  assess  whether  the  represented 
uncertainty  properly  characterizes  the  actual  uncertainty  of  the  space  object’s  orbital  state;  “most”  of  the 
particles  (blue  crosses)  should  be  contained  within  the  level  curves.  To  quantify  uncertainty  realism  more 
rigorously  and  hence  validate  the  prediction  steps  of  the  different  filters  under  consideration,  the  normalized 
L2  emor  [63]  is  studied. 
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For  functions  /,  g  :  A4  — >  M,  the  normalized  L->  error  between  /  and  g  is  the  scalar 


L2  (f,g) 
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where  ||  •  \\l2  is  the  L2  norm: 
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By  non-negativity  of  the  L2  norm,  L2  ^  0  with  equality  if  and  only  if  /  =  g  in  the  L2  sense.  By  the  triangle 
inequality,  L2  ^  1  with  equality  if  and  only  if  /  and  g  are  orthogonal  in  the  L2  sense. 

The  validation  tests  shown  in  Figure  7.10.2  generate  a  time  history  of  the  normalized  L2  error 


where  t  is  the  final  epoch  time.  Further,  papprox(w,  t)  represents  an  approximation  to  the  PDF  of  a  space 


object’s  orbital  state  at  time  t  (u  represents  equinoctial  orbital  elements)  as  computed  by  the  prediction  step 


of  the  GVM  filter,  EKF,  UKF,  or  a  Gaussian  sum  filter.  Moreover,  Pbaseiine(w,  t)  is  a  high-fidelity  approxi¬ 
mation  to  the  exact  state  PDF  which  serves  as  a  baseline  for  comparison.  This  baseline  is  computed  using  a 
high-fidelity  Gaussian  sum  using  the  GSF  of  Florwood  et  al.  [122].  Thus,  papprox  is  a  good  approximation 
to  the  actual  state  PDF  if  the  normalized  L2  error  between  it  and  the  baseline  pbaseiine  is  zero  or  “close  to 
zero”  for  all  time. 

7.10.2  Discussions 

As  described  in  Subsection  7. 10. 1 .2,  the  panels  in  Figure  7. 10. 1  show  the  evolution  of  a  space  object’s  orbital 
uncertainty  (with  initial  conditions  defined  in  Subsection  7.10.1.1)  computed  using  the  prediction  steps  of 
the  EKF,  UKF,  GVM  filter,  and  a  particle  filter.  Each  of  the  six  panels  shows  the  respective  level  curves 
in  the  plane  of  the  semi-major  axis  and  mean  longitude  coordinates  at  the  epochs  t  —  to  =  0,  0.5, 1,  2, 4,  8 
orbital  periods.  In  each  of  the  six  epochs,  the  level  curves  produced  by  the  GVM  filter  prediction  step  (shown 
in  shades  of  orange  and  red)  correctly  capture  the  actual  uncertainty  depicted  by  the  particle  ensemble.  Each 
set  of  level  curves  is  deduced  from  the  propagation  of  only  13  sigma  points  (corresponding  to  a  third-order 
GVM  quadrature  rule);  the  computational  cost  is  the  same  as  that  of  the  UKF.  For  the  UKF,  its  covariance 
(depicted  by  the  grey  ellipsoidal  level  curves)  is  indeed  consistent  (realistic)  in  the  sense  that  it  agrees  with 
that  computed  from  the  definition  of  the  covariance.  Thus,  in  this  scenario,  the  UKF  provides  “covariance 
realism”  but  clearly  does  not  support  “uncertainty  realism”  since  the  covariance  does  not  represent  the  actual 
banana-shaped  uncertainty  of  the  exact  PDF.  Further,  the  state  estimate  produced  from  the  UKF  coincides 
with  the  mean  of  the  true  PDF;  however  the  mean  is  displaced  from  the  mode.  Consequently,  the  probability 
that  the  object  is  within  a  small  neighborhood  centered  at  the  UKF  state  estimate  (mean)  is  essentially  zero. 
The  EKF,  on  the  other  hand,  provides  a  state  estimate  coinciding  closely  with  the  mode,  but  the  covariance 
tends  to  collapse  making  inflation  necessary  to  begin  to  cover  the  uncertainty.  In  neither  the  EKF  nor 
UKF  case  does  the  covariance  actually  model  the  uncertainty.  In  summary,  the  GVM  filter  prediction  step 
maintains  a  proper  characterization  of  the  uncertainty;  the  EKF  and  UKF  do  not. 
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Figure  7.10.1  The  uncertainty  of  a  space  object’s  orbital  state  at  different  epochs  t  —  to  computed  from  the 
prediction  steps  of  the  EKF,  UKF,  GVM  filter,  and  a  particle  filter.  Shown  are  the  respective  level  curves  in 
the  plane  of  the  semi-major  axis  and  mean  longitude  coordinates. 


(a)  t  -  t0  =  0 


(c)  t  —  to  =  1  orbital  period 


(b)  t  —  to  =  0.5  orbital  periods 


(e)  t  —  to  =  4  orbital  period 


(f)  t  —  to  =  8  orbital  periods 
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Figure  7.10.2  Plots  of  the  normalized  L2  error  using  different  methods  for  uncertainty  propagation. 


Time  (orbital  periods) 


Figure  7.10.2  shows  the  evolution  of  the  normalized  L2  error,  as  defined  in  Subsection  7.10.1.3,  com¬ 
puted  from  the  (prediction  steps  of  the)  UKF,  EKF,  the  Gaussian  sum  filter  (GSF)  of  Fiorwood  et  al.  [122] 
with  N  =  17  and  A'  =  19  components,  and  the  GVM  filter.  Uncertainty  propagation  using  the  UKF  and 
EKF  quickly  break  down,  but  accuracy  can  be  improved  by  increasing  the  fidelity  of  the  Gaussian  sum. 
The  normalized  L2  errors  produced  from  the  GVM  filter  prediction  step  lie  between  those  produced  from 
the  17  and  49-term  Gaussian  sum.  It  is  noted  that  the  17-term  Gaussian  sum  requires  the  propagation  of 
17  x  13  =  221  sigma  points;  the  GVM  distribution  only  requires  13.  If  one  deems  a  normalized  L2  error  of 
L2  =  0.05  to  signal  a  breakdown  in  accuracy,  then  the  UKF  and  EKF  prediction  steps  first  hit  this  threshold 
after  about  one  orbital  period.  By  examining  when  the  normalized  L2  error  first  crosses  0.05  for  the  GVM 
filter  prediction,  it  is  seen  that  the  GVM  filter  prediction  step  can  faithfully  propagate  the  uncertainty  for 
about  8  times  longer  than  when  using  an  EKF  or  UKF. 

These  results  also  suggest  what  could  be  achieved  if  the  orbital  state  uncertainty  is  represented  as  a 
mixture  of  GVM  distributions,  in  analogy  to  the  Gaussian  sum  (mixture).  As  a  single  GVM  distribution 
achieves  accuracy  (in  the  L2  sense)  between  the  17  and  49-term  Gaussian  mixture,  it  is  hypothesized  that  a 
GVM  mixture  would  require  about  95%  fewer  components  than  a  Gaussian  mixture  to  achieve  comparable 
accuracy  and  uncertainty  realism.  Thus,  one  can  extend  the  validity  over  which  the  uncertainty  is  propagated 
without  the  ballooning  cost  of  doing  so.  Extending  the  work  of  this  paper  to  GVM  mixtures  is  future 
research. 

7.11  Polynomial  Chaos 

Polynomial  chaos  (PC)  approximates  the  solution  of  a  stochastic  differential  equation  by  projecting  it  onto  a 
basis  of  orthogonal  polynomials.  The  work  in  [286]  first  proposed  this  type  of  approximation,  with  methods 
based  on  Hermite  polynomial  chaos  more  recently  established  in  [97,  98],  among  other  works,  and  general- 
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ized  to  other  types  of  orthogonal  polynomials  [295].  We  note  that  the  term  “polynomial  chaos”  was  adopted 
before  the  modern  definition  of  chaos  (implying  a  sensitive  dependence  on  initial  conditions).  In  the  context 
of  PC,  chaos  refers  to  random. 

Propagation  of  orbit  state  uncertainty  using  polynomial  chaos  leverages  a  stochastic  ordinary  differential 
equation  (SODE)  description  of  the  dynamics,  i.e.,  it  considers  one  or  more  inputs  to  the  propagator  as  a  ran¬ 
dom  variable  with  some  a  priori  knowledge  of  its  distribution.  The  PC-based  approximation  of  this  SODE, 
known  as  a  polynomial  chaos  expansion  (PCE),  describes  the  propagated  solution  as  a  function  of  its  random 
inputs  These  random  inputs  correspond  to,  for  example,  initial  position,  velocity,  and  force  model  param¬ 
eters.  Using  a  PCE  to  quantify  uncertainty  differs  from  methods  directly  propagating  a  probability  density 
function  (PDF)  in  that  the  polynomial  surrogate  describes  the  sensitivity  of  the  solution  to  the  stochastic 
inputs.  Information  on  the  PDF  may  then  be  determined  via  analytic  analysis  or  random  evaluations  of  the 
PCE  solution.  Polynomial  chaos  is  an  active  area  of  research  in  the  applied  mathematics  community,  with 
demonstrated  uses  for  uncertainty  quantification  in  fluid  dynamics  [173,  295],  solid  mechanics  [97,  98],  and 
many  other  applications.  In  the  context  of  astrodynamics,  PCE-based  propagation  of  uncertainty  has  been 
demonstrated  for  orbit  uncertainty  propagation  given  a  probabilistic  description  of  the  initial  state  [149],  to 
estimate  the  probability  of  collision  between  two  spacecraft  [148,  151],  and  for  characterizing  the  sensitivity 
of  atmospheric  density  to  the  model  inputs  [270]. 

Figure  7.11.1  Outline  of  non-intrusive  solution  generation  process  for  a  PCE 


In  the  context  of  stochastic  differential  equations,  a  solution  of  interest  X(t ,  r,  £)  is  a  function  of  the 
random  inputs  £  e  K.rf  and  can  be  a  function  of  time  t  and/or  position  r.  The  PCE  solution  is  then 

X{t,r,i)=  Y  Ca(t,r)il;a(£),  (7.11.1) 

q£Nq 

where  Nq  :=  {(an,  •  •  •  ,  av/)  :  at  6  N  IJ  {0}}  is  the  set  of  multi-indices  of  size  d  defined  on  non-negative 
integers.  The  basis  functions  {ipa($)}  a1'6  multi-dimensional  spectral  polynomials  of  maximum  degree  p, 
referred  to  as  the  “polynomial  chaos,”  that  are  orthogonal  with  respect  to  the  joint  probability  measure  of 
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Details  on  the  selection  of  the  polynomial  basis  may  be  found  in  the  literature  (e.g.,  see  [295]).  The  exact 
vector  of  generalized  Fourier  coefficients  cn(t.  r)  in  Equation  (7.11.1),  referred  to  as  the  PC  coefficients, 
are  computed  by  the  projection  of  X(t ,  r,£)  onto  each  basis  function  and  may  be  approximated 

by  sampling  an  existing  solver  as  a  black-box  [98,  172,  294].  The  PCE  solution  is  then  the  best  linear 
approximation  of  X(t,r,£)  in  a  finite  dimensional  space  spanned  by  multi-dimensional  polynomials  in 
£.  This  general  process,  dubbed  a  non-intrusive  method  since  it  treats  a  given  propagator  as  a  black  box, 
is  outlined  in  Figure  7.11.1.  Example  non-intrusive  methods  of  generating  a  PCE  approximation  include, 
but  are  not  limited  to:  least  squares  regression  [128],  pseudo-spectral  collocation  [172,  294],  Monte  Carlo 
sampling  [172],  and  compressive  sampling  [71].  For  the  case  of  orbit  state  uncertainty  propagation,  the 
equations  of  motion  are  described  by  a  SODE  and  the  output  (often  position  and  velocity)  state  X(t,£). 
Hence,  for  orbit  state  uncertainty  propagation,  the  PCE  approximation  is 

X(t,t)=  E  (0,  (7-11-2) 

CX^\pd 

where 

A P,d  =  jo:  G  Nq  :  ^  p,  ||a||0  ^  dj  ,  (7.11.3) 

and  || a || o  denotes  the  number  of  non- zero  elements  in  a.  The  maximum  polynomial  degree  p  is  selected 
based  on  accuracy  requirements,  computation  time  limitations,  etc.,  and  (along  with  d)  determines  the  num¬ 
ber  of  elements  in  Equation  7.1 1.2. 

As  an  example  of  the  use  of  PC,  consider  the  function 

/(0=£2  +  £-  1  (7-11-4) 

where  the  input  £  is  a  random  variable  with  zero  mean  and  unit  variance.  Since  there  is  only  one  random  in¬ 
put,  then  d  =  1.  Although  not  depicted  here,  the  PDF  describing  /  is  non-Gaussian.  When  using  regression 
to  generate  a  PCE  approximation  of  this  function  with  a  maximum  polynomial  degree  p  =  2,  we  instead 
express  Equation  (7.1 1.2)  as  the  linear  system 


/(£) 


MO  MO  MO 


co 

Cl 

C2 


(7.11.5) 


Since  the  random  input  £  has  a  standard  normal  distribution,  then  the  basis  functions  ipQ  are  Hermite  poly¬ 
nomials  each  with  degree  a  [295].  In  this  example,  we  will  generate  a  solution  for  the  coefficients  co,  ci, 
and  C2  using  least-squares  regression.  Given  M  A  3  random  realizations  of  £,  we  have 


M6) 

M6) 

MO) 

7(6)' 

MO) 

MO) 

7  (£2) 

co 

Cl 

= 

/( 6) 

_M6w) 

Vh  (Or) 

MOr). 

_C2_ 

DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


87 


7. 12.  SPARSE  GRID  QUADRATURE  APPROACHtmAfflERTlJimE&MINimminPNN&mmiNTY 


which  may  be  solved  using  the  normal  equation  solution  of  the  least  squares  estimator.  Given  M  3,  this 
yields  a  reasonable  approximation  of  the  true  solution 

co  =  0,  Cl  =  1,  02  =  1.  (7.11.7) 

In  the  context  of  Figure  7.11.1,  the  inputs  £  are  generated  from  a  given  pseudo-random  number  generator 
(pRNG),  the  propagation  of  the  sample  is  the  evaluation  of  Equation  (7.11.4)  for  a  given  £*,  and  the  gen¬ 
eration  of  the  PCE  is  the  estimation  of  the  coefficients  using  least  squares.  In  this  case,  conversion  of  the 
pRNG  output  to  samples  based  on  the  a  priori  PDF  is  not  required,  but  it  would  be  necessary  when  given  an 
initial  orbit  state  PDF  and  an  orbit  propagator. 

The  key  advantages  of  these  PC-based  techniques  are: 

•  fast,  up  to  exponential,  mean-squares  convergence  of  the  expansion  with  respect  to  the  order  of  the 
polynomial  basis  even  when  u(t,  r,  £)  is  highly  non-Gaussian, 

•  construction  of  an  explicit  functional  representation  (i.e.,  response  surface)  of  the  solution  of  interest 
with  respect  to  the  random  inputs  £,  and 

•  the  combination  of  random  inputs  £  each  described  by  different  PDF’s  (i.e.,  Generalized  Polynomial 
Chaos). 

Disadvantages  include: 

•  the  curse  of  dimensionality  that  causes  problems  for  most  methods  of  uncertainty  quantification,  and 

•  some  applications  require  some  post-processing  to  analyze  the  a  posteriori  PDF. 

It  is  noted  that  the  effects  of  the  curse  of  dimensionality  may  be  mitigated  through  various  means  (e.g., 
separated  representations  and  compressive  sampling),  which  constitutes  an  active  area  of  research  (e.g., 
see  [70-72,  213,  296]  and  the  references  therein).  However,  the  use  of  such  methods  remains  somewhat 
problem  dependent,  but  generating  a  PCE  using  compressive  sampling  has  already  been  demonstrated  for 
propagating  uncertainty  after  a  spacecraft  maneuver  [151].  Additionally,  [23]  used  separated  representations 
(similar  to  a  PCE)  to  propagate  orbit  state  uncertainty,  but  some  work  remains  to  demonstrate  its  efficacy 
for  mitigating  the  curse  of  dimensionality  for  such  a  problem.  PC  and  Gaussian  mixtures  models  have  also 
been  combined  showing  benefits  that  neither  alone  can  provide  [271]. 

7.12  Sparse  Grid  Quadrature  Approach  to  Orbit  Uncertainty  Propagation 

Sparse  grid  quadrature  is  a  deterministic  numerical  approach  to  computing  the  moments  of  the  orbit  state, 
for  example,  mean,  covariance,  skewness,  and  kurtosis.  The  sparse  grid  quadrature  approach  is  applicable  to 
the  orbit  uncertainty  propagation  problem  in  which  the  orbit  dynamics  can  be  written  as  xt  =  0(t;  xq,  a), 
where  t  denotes  time,  xt  and  xo  denote  the  state  at  time  t  and  initial  time  to,  respectively,  a  denotes  the 
model  parameter  vector,  and  0  denotes  the  nonlinear  mapping  from  to  to  t  [210,  257].  The  orbit  state 
can  be  either  position  and  velocity  or  orbital  elements.  In  coupled  attitude/orbit  uncertainty  propagation, 
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the  state  may  include  attitude  and  angular  velocity  as  well  [257].  The  state  and  model  parameters  can  be 
either  Gaussian  or  non-Gaussian.  The  approach  treats  the  mapping  0  as  a  “black  box”  and  can  therefore 
be  used  with  any  force  model.  Applications  of  this  approach  to  orbit  uncertainty  propagation  can  be  found 
in  [210,  257], 

The  sparse  grid  quadrature  approach  to  orbit  uncertainty  propagation  is  based  on  the  fact  that  moments  of 
the  orbit  state  are  mathematical  expectations  of  nonlinear  transformations  of  the  initial  orbit  state  and  model 
parameters  and  therefore  can  be  computed  using  a  quadrature  (or  cubature)  method.  For  example,  the  state 
mean  xt  =  E[<fi(t,  xo,  a)]  ~  Y^iLi  wi  xo'\  ce^),  where  the  expectation  is  taken  with  respect  to  the 
joint  probability  density  function  of  x$  and  a,  [x\^T ,  a^')T]T  are  quadrature  points  in  the  joint  space  of  xq 
and  a,  Wi  are  the  associated  weights,  and  N  is  the  number  of  quadrature  points  [210,  257].  The  superscript 
T  denotes  transpose.  Fligher  moments  can  be  computed  in  a  similar  manner.  An  re-dimensional  sparse  grid 
quadrature  is  constructed  from  linear  combinations  of  tensor  products  of  one-dimensional  quadratures  by 
using  Symolyak  rule.  The  one-dimensional  quadratures  are  usually  from  the  family  of  Gaussian  quadratures, 
for  example,  Gaussian-Flermite  quadrature  for  Gaussian  distributions  and  Gaussian-Legendre  quadrature 
for  uniform  distributions  [210].  Moment  matching  can  also  be  used  to  determine  the  points  and  weights  of 
one-dimensional  quadratures  [142].  A  sparse  grid  quadrature  can  be  anisotropic,  which  means  that  more 
quadrature  points  are  located  along  certain  directions  than  along  other  directions  [141].  The  sparse  grid 
quadrature  differs  from  other  quadratures  only  in  the  quadrature  points  and  weights. 

If  the  mapping  0(t;  xq,  a )  is  polynomial  in  xq  and  a,  the  sparse  grid  quadrature  with  sufficient  number 
of  quadrature  points  can  compute  any  moments  of  xt  exactly  [210].  This  makes  the  sparse  grid  quadrature 
approach  very  appealing  to  moment  computation  for  polynomial-like  systems.  The  computational  com¬ 
plexity  of  the  sparse  grid  quadrature  is  proportional  to  the  number  of  quadrature  points  N,  a  polynomial 
function  of  the  dimension  of  the  joint  vector  of  x$  and  a  [210].  The  widely  used  unscented  transform 
of  the  unscented  Kalman  filter  or  sigma-point  filter  can  be  viewed  as  a  low-accuracy  sparse  grid  quadra¬ 
ture  [140,  142].  In  addition  to  orbit  uncertainty  propagation,  nonlinear  filters  based  on  sparse  grid  quadra¬ 
ture  have  been  developed  and  shown  to  be  more  accurate  and  have  better  convergence  properties  than  the 
extended  Kalman  filter  or  the  unscented  Kalman  filter  in  the  presence  of  large  initial  uncertainty  [140-142]. 
The  cubature  Kalman  filter  [143]  is  also  closely  related  to  the  sparse  grid  quadrature  filter  [144].  The  sparse 
grid  method  can  take  into  consideration  the  discrete-time  process  noise  by  a  linear  transformation  of  the 
quadrature  points  [52]. 

Advantages  The  sparse  grid  method  is  an  efficient  and  easy-to-implement  method  for  computing  the 
moments  of  the  orbit  state.  It  only  requires  forward  orbit  propagation  and  the  number  of  orbit  propaga¬ 
tions  is  polynomial  (instead  of  exponential)  in  the  dimension  of  the  state  and  model  parameters.  It  can  be 
used  for  a  large  class  of  orbit  uncertainty  propagation  problems,  where  the  uncertainty  can  be  Gaussian  or 
non-Gaussian  and  the  state  can  be  Cartesian  coordinates  or  orbital  elements.  The  accuracy  of  the  sparse 
grid  method  is  high  when  the  nonlinear  mapping  is  polynomial  like.  One  such  example  is  the  mean  and 
covariance  propagation  of  an  LEO  object  for  15  orbital  periods  [210]. 

Disadvantages:  The  sparse  grid  method  itself  cannot  provide  a  parametric  model  of  the  probability 
density  function.  The  accuracy  of  the  sparse  grid  method  depends  on  whether  the  nonlinear  mapping  is 
polynomial-like.  When  the  mapping  is  significantly  dissimilar  to  a  polynomial  function,  the  accuracy  of 
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the  sparse  grid  method  degrades  and  is  no  better  than  the  Monte  Carlo  or  quasi  Monte  Carlo  method  [210]. 
(Moments  computed  using  state  transition  tensors,  generalized  polynomial  chaos,  and  conjugate  unscented 
transform  have  similar  accuracy  degradation  when  the  polynomial  approximation  of  the  nonlinear  mapping 
is  invalid.)  The  sparse  grid  method  cannot  guarantee  that  all  the  weights  are  positive.  This  is  no  problem 
when  the  nonlinear  mapping  is  polynomial  or  polynomial-like,  but  when  the  mapping  is  far  from  polynomial, 
that  is,  when  the  applicability  of  the  sparse  grid  method  is  questionable,  the  computed  variance  or  other  even 
moments  may  be  negative  [210]. 

7.13  Particle  Filters 

Particle  filters  are  sequential  Monte  Carlo  methods  based  on  the  representation  of  an  arbitrary  PDF  by  a  set 
of  independent  and  identically  distributed  random  samples.  They  can  be  applied  to  any  nonlinear  and/or 
non-Gaussian  state  space  model;  however,  they  are  computationally  intensive.  Particle  filters  were  exten¬ 
sively  developed  in  the  nineties  [21,  228]  and  remain  a  applicable  approach  to  propagating  uncertainty  in 
astrodynamics  [174,  193,  194].  Although  computationally  expensive,  a  key  advantage  is  the  representation 
of  general  PDFs.  In  addition,  particle  filters  are  easily  parallelized  [20,  242], 

Particle  filters  utilize  Monte  Carlo  sampling  to  approximate  the  posterior  PDF  via  random  samples  with 
appropriately  chosen  weights.  Monte  Carlo  simulation  is  applied  in  conjunction  with  importance  sampling 
to  allow  for  state  estimation  using  weights  computed  from  a  sampled  posterior  density.  Importance  sampling 
computes  the  weights  of  generated  samples  and  is  deemed  necessary  because  it  is  not  possible,  in  general, 
to  effectively  sample  the  posterior  distribution  due  to  its  multivariate,  nonstandard,  and  only  known  up  to  a 
certain  order  nature.  Importance  sampling  requires  the  user  to  define  the  analytic  character  of  the  posterior 
state  PDF,  known  as  the  importance  or  proposed  density  [228].  The  filter  designer  proposes  the  importance 
density  and  the  number  of  particles  to  be  utilized  by  the  filter  to  produce  an  accurate  and  consistent  estimate 
while  balancing  computational  burden.  One  of  the  over-looked  requirements  for  implementation  of  particle 
filtering  is  that  the  density  of  instantaneously  unobserved  variables  must  be  dictated.  For  example,  if  only 
position  level  measurements  are  available,  the  velocity  distribution  must  be  given  so  that  the  filter  can  com¬ 
pute  samples.  If  an  incorrect  form  is  given  for  instantaneous  unobserved  state  variables,  degeneracy  will 
arise  faster  and  more  resampling  steps  are  required.  If  the  posterior  PDF  is  not  known  exactly,  as  is  common 
in  most  problems,  a  posterior  approximation  must  be  used  rendering  the  PF  suboptimal.  Commonly,  the 
importance  density  is  selected  to  fit  the  assumption  that  state  variable  uncertainties  arc  of  Gaussian  char¬ 
acter  with  a  given  mean  and  covariance.  The  Gaussian  assumption  for  the  importance  density  produces  a 
suboptimal  filter  for  nonlinear  problems,  even  if  the  Gaussian  assumption  is  valid  for  a  certain  state  variable, 
it  may  not  be  valid  for  others.  Using  the  Gaussian  distribution  assumption,  the  importance  density  becomes 
equivalent  to  the  prior.  If  the  posterior  PDF  is  relatively  well  known,  the  number  of  samples  must  be  rather 
large  so  that  the  importance  density  approaches  the  true  posterior  PDF. 

If  the  chosen  importance  density  is  not  exactly  the  posterior  distribution,  the  variance  of  the  importance 
weights  has  been  shown  to  increase  as  the  number  of  recursive  steps  increases  leading  to  only  one  particle 
possessing  a  nonzero  weight.  This  is  known  as  degeneracy  and  resampling  is  carried  out  to  combat  this 
problem  [228].  The  resampling  step  eliminates  samples  with  low  importance  and  increases  the  number  of 
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samples  with  high  importance  to  avoid  excessive  computational  resources  for  near  zero  weights.  However, 
resampling  requires  a  measure  of  degeneracy,  such  as  the  inverse  of  the  sum  of  the  square  weights,  and  a 
user-defined  threshold  for  triggering  the  resampling  procedure.  Accordingly,  appropriate  selection  of  the 
importance  density  is  crucial  to  particle  filter  operation  since  incorrect  selection  can  result  in  divergence  or 
excessive  degeneracy.  Refinement  methods  have  been  introduced  to  increase  the  accuracy  of  the  importance 
density  in  various  ways:  (a)  using  intermediate  densities  between  time  steps  to  re-weight  the  particles  which 
are  resampled,  (b)  using  the  measurement  at  time  K  to  refine  the  particles  at  time  K  —  1  before  propagation, 
or  (c)  applying  an  EKF  or  UKF  to  generate  a  Gaussian  approximation  for  the  importance  density  [228]. 

Sample  diversity  is  an  important  concern  since  degeneracy  is  inherent  to  sequential  importance  sampling 
particle  filters  and  resampling  can  cause  diversity  loss  among  particles.  Other  methods  such  as  regulariza¬ 
tion  or  the  Markov  Chain  Monte  Carlo  (MCMC)  move  step  have  been  implemented  to  maintain  diversity, 
which  can  be  hard  to  accomplish  especially  for  systems  with  little  or  no  process  noise.  The  regularized  PF 
combats  loss  of  diversity  by  jittering  the  particles  selected  from  the  importance  density  by  a  proposed  kernel 
density.  Since  particle  jittering  can  cause  divergence  from  the  true  posterior,  addition  of  a  MCMC  move 
step,  utilizing  the  Metropolis-Hastings  acceptance  probability  dictating  jittering  acceptance,  can  improve 
operation.  If  the  propagation  process  equations  have  little  to  no  process  noise,  this  will  result  in  significant 
resampling  because  the  sampled  a  posteriori  density  particles  will  not  span  enough  of  the  space  to  maintain 
diversity  for  long  compared  with  a  propagation  process  that  is  augmented  with  process  noise. 

7.14  Differential  Algebra 

The  basic  idea  behind  differential  algebra  (DA)  is  to  bring  the  treatment  of  functions  and  the  operations 
on  them  to  a  computer  environment  in  a  similar  manner  as  the  treatment  of  real  numbers  [35,  pp  82-96]. 
Referring  to  Figure  7.14.1,  consider  two  real  numbers  a  and  b.  Their  transformation  into  the  floating-point 
(FP)  representation,  a  and  b,  respectively,  is  performed  to  operate  on  them  in  a  computer  environment. 
Then,  given  any  operation  *  in  the  set  of  real  numbers,  an  adjoint  operation  ©  is  defined  in  the  set  of  FP 
numbers  so  that  the  diagram  in  Figure  7.14.1  commutes.  (The  diagram  commutes  approximately  in  practice 
due  to  truncation  errors.)  Consequently,  transforming  the  real  numbers  a  and  b  into  their  FP  representation 
and  operating  on  them  in  the  set  of  FP  numbers  returns  the  same  result  as  carrying  out  the  operation  in 
the  set  of  real  numbers  and  then  transforming  the  achieved  result  in  its  FP  representation.  In  a  similar 
way,  given  two  k-differentiable  functions,  /  and  g,  in  n  variables,  DA  operates  on  them  using  their  A  th- 
order  Taylor  expansions,  F  and  G,  respectively.  Therefore,  the  transformation  of  real  numbers  in  their 
FP  representation  is  now  replaced  by  the  extraction  of  the  fcth-order  Taylor  expansions  of  /  and  g  in  the 
computer  environment.  For  each  operation  in  the  space  of  fc-differentiable  functions,  an  adjoint  operation  in 
the  space  of  Taylor  polynomials  is  defined,  so  that  the  corresponding  diagram  commutes;  that  is,  extracting 
the  Taylor  expansions  of  /  and  g  and  operating  on  them  in  the  space  of  Taylor  polynomials  using  DA  returns 
the  same  result  as  operating  on  /  and  g  in  the  original  space  and  then  extracting  the  Taylor  expansion  of  the 
resulting  function.  The  proper  implementation  of  DA  in  a  computer  allows  the  user  to  efficiently  compute 
the  Taylor  coefficients  of  any  coded  function  up  to  a  specified  order  k. 

The  availability  of  such  high  order  expansions  can  be  exploited  when  uncertainties  are  propagated 
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Figure  7.14.1  Floating-point  representation  of  real  numbers  (left)  and  algebra  of  Taylor  polynomials  (right). 


a,  6  E  R 


T 


a,  6  E  FP 


a*b 


T 


a®b 


f,g 


V 


F,  G 


f  *g 


V 


t 

F  ®  G 


through  nonlinear  functions  or  dynamical  models.  Without  loss  of  generality,  consider  the  initial  value 
problem 

dx(t) 

— 77-  =  ,<x,t),x  (to)  =  x0,  (7.14.1) 

at 

where  x  is  the  state  vector  and  a  is  the  model  parameter  vector.  The  associated  flow  is  denoted  by 
4>(t;xo,a).  The  /cth -order  Taylor  expansion  of  the  flow  at  any  time  t  with  respect  to  initial  conditions 
and  model  parameters  can  be  obtained  by  initializing  xq  and  a  as  DA  variables,  and  carrying  out  all  the 
operations  of  an  integration  algorithm  in  the  DA  framework.  The  resulting  polynomial  is  a  /cth -order  map 
linking  the  domain  of  initial  conditions  and  model  parameters  to  the  manifold  of  attainable  states  at  time  t. 

The  polynomial  map  can  be  used  to  efficiently  propagate  uncertainties  through  different  methods.  A 
first  method  consists  of  running  Monte  Carlo  simulations  on  the  map  rather  than  on  the  original  flow.  In  this 
way,  it  is  possible  to  substitute  thousands  of  point-wise  numerical  integrations  required  for  classical  Monte 
Carlo  simulations  with  an  equal  number  of  map  evaluations  (i.e.,  fast  polynomial  evaluations).  Applications 
of  this  method  to  the  analysis  of  asteroid  close  encounters  with  Earth  and  to  impact  probability  computation 
between  Earth-orbiting  objects  can  be  found  in  [18,  208, 209].  Secondly,  DA  can  be  exploited  to  compute  the 
moments  of  the  transformed  probability  density  function  (e.g.,  mean,  covariance,  skewness,  and  kurtosis)  by 
applying  the  expectation  operation  to  the  polynomial  maps  [266,  267],  or  to  analytically  map  the  probability 
density  function  by  computing  the  expansion  of  the  determinant  of  the  Jacobian  of  the  transformation  [290]. 
Lastly,  polynomial  bounders  can  be  applied  on  the  polynomial  map  to  estimate  the  maximum  size  of  the 
propagated  uncertainty  set  along  each  coordinate  [180]. 

Advantages 

Differential  algebra  is  an  automatic  procedure  that  allows  the  expansion  of  the  flow  of  ODE  at  arbitrary  or¬ 
der.  Thus,  the  approach  is  suitable  for  dealing  with  highly  nonlinear  dynamics,  where  linearized  approaches 
fail  to  deliver  an  accurate  description  of  the  evolution  of  the  uncertainty  set.  In  addition,  as  the  flow  ex¬ 
pansion  is  obtained  with  an  algebra  of  Taylor  polynomials,  DA  method  does  not  involve  the  integration  of 
variational  equations  and  its  implementation  is  essentially  problem  independent.  DA  can  be  implemented 
very  efficiently  in  a  computer  environment,  thus  it  is  possible  to  obtain  high  order  expansions  of  the  flow  in 
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a  limited  amount  of  time.  As  an  example,  a  sixth  order  expansion  of  the  flow  of  a  restricted  n-body  problem 
takes  on  average  only  10  times  more  than  a  simple  point- wise  integration.  This  shows  that  a  DA-based 
Monte  Carlo  simulation  can  be  orders  of  magnitude  faster  than  a  classical  one,  when  a  significant  number  of 
particles  need  to  be  propagated.  The  DA  approach  for  uncertainty  propagation  is  also  independent  from  the 
initial  statistics.  In  case  different  initial  statistics  need  to  be  analyzed,  the  same  polynomial  map  (obtained 
with  a  single  DA  integration)  can  be  used.  The  same  consideration  applies  to  model  parameters.  Finally, 
as  the  flow  expansion  is  analytical,  an  analytic  framework  is  delivered.  The  use  of  tools  for  inverting  and 
composing  polynomials,  as  well  as  integrating  and  differentiating  them,  allows  the  user  to  build  algorithms 
to  solve  complex  problems  that  may  also  involve  constraint  satisfaction. 

Disadvantages 

DA  approach  is  based  on  Taylor  expansions.  Thus,  the  problem  to  be  dealt  with  must  be  well  behaved. 
Discontinuous  dynamical  models  (e.g.,  eclipses  or,  in  general,  if  statements)  are  a  major  limitation  for  the 
method,  and  complicated  ad  hoc  procedures  are  needed  to  deal  with  these  conditions  (when  possible).  In 
addition,  the  accuracy  of  the  method  tends  to  decrease  drastically  when  the  uncertainty  domain  becomes 
too  stretched  in  one  or  more  directions.  This  can  be  due  to  one  or  a  combination  of  the  following  causes: 
high  nonlinearity  of  the  dynamics,  large  initial  uncertainty  sets,  and  long  term  propagations.  Thus,  it  is 
necessary  to  estimate  the  convergence  radius  of  the  Taylor  expansion  in  order  to  guarantee  the  fulfillment 
of  the  accuracy  requirements.  In  order  to  address  this  problem,  a  method  referred  to  as  automatic  domain 
splitting  has  been  recently  proposed.  The  underlying  idea  is  to  split  the  initial  domain  into  manageable 
subdomains  over  which  the  Taylor  expansion  shows  good  convergence  properties  [291].  This  results  in  the 
need  of  managing  multiple  Taylor  expansions,  which  can  be  an  issue  when  many  splits  occur. 

7.15  Numerical  Integrators  for  Orbit  State  and  Uncertainty  Propagation 

The  subject  of  the  numerical  integration  of  ordinary  differential  equations  has  an  extensive  history  and 
rich  development  from  which  one  can  draw  for  the  specialization  of  methods  and  the  development  of  new 
ones  for  propagating  astrodynamic  states  and  their  associated  uncertainty.  One  can  use  virtually  any  of 
these  well  developed  methods  for  the  purposes  of  orbit  propagation.  Thus,  one  might  ask  “Why  is  there  a 
need  for  further  development?”  The  answer  lies  in  the  current  and  future  needs  given  the  large  number  of 
space  objects  expected  in  the  future  and  the  need  to  propagate  orbits  and  their  uncertainty  as  efficiently  as 
possible.  One  of  the  requirements  is  that  whatever  method  is  used,  the  numerical  error,  i.e.,  the  discretization 
and  truncation  errors,  in  the  orbital  propagators  should  remain  significantly  below  the  uncertainty  in  the 
dynamical  model.  There  are  several  new  orbital  propagators  that  respond  to  this  need,  many  of  which 
are  briefly  reviewed  here.  Recent  developments  have  focused  on  the  efficient  use  of  gravity  models,  highly- 
efficient  integrators  for  specific  orbit  types,  error  estimation  and  stepsize  control  to  automate  the  propagation 
of  any  orbit  type,  parallelization  to  reduce  computational  costs,  large  step  sizes,  e.g.,  half  an  orbit  or  more, 
allowed  by  many  of  the  newer  propagators,  and  the  propagation  of  an  ensemble  of  orbits  to  support  particle  - 
and  sigma-point-based  approaches  to  the  propagation  of  uncertainty. 
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Many  numerical  integration  methods  have  been  specialized  for  propagating  orbits,  as  surveyed  in  Mon- 
tenbruck  [206],  Montenbruck  and  Gill  [207],  and  Jones  and  Anderson  [146].  Current  state-of-the-art  nu¬ 
merical  integrators  used  for  orbit  propagation  include  Dormand-Prince  8(7)  (DP8),  Runge-Kutta-Nystrom 
12(10)  (RKN12),  Adams-Bashforth-Moulton  (ABM),  and  Gauss-Jackson  (GJ).  Both  DP8  and  RKN12  are 
high-order  explicit  Runge-Kutta  methods  that  use  variable-stepsize  error  control  (Iserles  [137],  Butcher 
[44],  Hairer,  Norsett  and  G.  Wanner  [107]).  However,  RKN12  is  an  integration  method  that  cannot  han¬ 
dle  velocity-dependent  forces,  and  so  its  use  for  general  orbit  propagation  is  limited.  ABM  and  GJ  are 
predictor-corrector  methods  that  utilize  an  explicit  method  for  the  prediction  and  an  implicit  method  for 
the  correction.  Fixed-step  implementations  of  GJ  (using  judiciously  chosen  Sundman  regularizations  for 
propagating  highly-elliptic  orbits)  are  widely  used  in  numerical  integration  problems  for  astrodynamics  and 
dynamical  astronomy.  For  example,  an  eighth-order  GJ  algorithm  has  been  used  for  space  surveillance  since 
the  1960s  (Berry  and  Healy  [32]).  It  is  worth  noting  that  although  GJ  will  reduce  its  step  size  (typically  by 
factors  of  two)  to  ensure  convergence  of  a  given  time  step;  however,  this  does  not  imply  that  GJ  is  a  variable- 
step  integration  method  as  discussed  below.  JPL  uses  a  variable-step,  variable-order  multi-step  propagator 
due  to  Keogh  [169,  170]  which  is  reported  to  have  better  stability  characteristics  than  the  aforementioned 
ABM  method. 

More  recently,  Bradley,  Jones,  Beylkin,  and  Axelrad  [39]  studied  the  use  of  fixed-step  implicit  Runge- 
Kutta  (IRK)  methods  for  orbit  propagation  based  on  Gauss-Legendre  quadrature.  Bai  and  Junkins  [22]  in¬ 
vestigated  a  related  scheme  based  on  Gauss-Chebyshev-Lobatto  quadrature  and  fixed-point  iteration,  which 
they  coined  modified  Chebyshev-Picard  iteration  (MCPI).  Gauss-Legendre  IRK  (GL-IRK)  and  Gauss-Chebyshev 
IRK  methods  are  of  particular  interest  because,  unlike  the  above  explicit  methods,  they  are  parallelizable, 
thus  amenable  to  high-performance  computing,  and  A-stable  at  all  orders,  thus  allowing  for  larger  (and 
fewer)  time  steps  to  be  taken  without  sacrificing  stability  (Iserles  [137],  Butcher  [44],  Hairer  and  Wanner 
[108]*.  The  caveat  is  that  a  nonlinear  system  of  equations  must  be  solved  via  iterative  methods  at  each  time 
step.  While  this  is  a  clear  disadvantage  for  implicit  methods  in  general,  for  the  perturbed  perturbed  two- 
body  problem,  one  may  use  approximate  analytical  solutions  to  warm-start  the  iterations,  leading  to  faster 
convergence. 

By  tuning  the  number  of  time  steps,  the  number  of  stages  (nodes)  per  time  step,  and  the  convergence 
criteria  (iteration  tolerance  for  a  given  time  step),  these  authors  demonstrated  that  IRK  methods  can  be  made 
both  precise  and  efficient,  at  least  for  nearly-circular  orbits,  based  on  the  number  of  force-model  evaluations 
required  to  propagate  the  orbit  -  in  a  serial  computing  environment.  In  Bai  and  Junkins’  study  [22],  zonal 
harmonics  in  the  Earth’s  gravity  up  to  degree  five  were  considered;  higher-fidelity  gravitational  force  models 
were  considered  by  Beylkin  and  Sandberg  [36]  and  Bradley,  Jones,  Beylkin,  Sandberg  and  Axelrad  [39,  40], 
specifically,  zonal,  tesseral,  and  sectorial  harmonics  up  to  degree  and  order  70. 

Variable-step  methods,  on  the  other  hand,  estimate  the  numerical  truncation  err  or  and  control  it  by  adapt¬ 
ing  the  stepsize.  In  addition  to  rejecting  or  accepting  the  current  stepsize  based  on  meeting  the  prescribed 

^Recent  work  by  Beylkin,  Sandberg,  Bradley,  Jones  and  Axelrad  [36,  40]  suggests  that  band-limited  collocation-based  IRK 
methods  are  more  efficient  (for  a  large  number  of  nodes)  than  their  Gauss-Legendre  and  Gauss-Chebyshev  counterparts,  while  also 
being  parallelizable  and  “numerically”  A-stable. 

3  In  the  perturbed  two-body  problem,  evaluation  of  the  force  models  (the  Earth  gravity  model  in  particular)  is  the  dominant  cost 
of  orbit  propagation.  The  number  of  force-model  evaluations  can  thus  be  used  as  a  crude  measure  of  propagator  performance. 
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error  tolerance  and  convergence  of  the  nonlinear  equations,  these  methods  also  predict  the  stepsize  for  the 
next  time  step.  This  is  different  from  adjusting  the  stepsize  to  achieve  convergence  of  the  above  nonlinear 
equations  in  the  implicit  methods.  While  there  is  a  computational  price  for  maintaining  control  over  the 
truncation  error,  these  variable-step  methods  with  error-stepsize  control  generally  lead  to  a  more  efficient 
method  suitable  for  all  regimes  of  space  without  modification  [15].  Variable-step  GL-IRK-based  orbital 
propagators,  such  as  those  developed  by  Jones  [145],  and  Aristoff,  Horwood,  and  Poore  [11,  14,  15],  on  the 
other  hand,  can  be  considered  self-tuning.  The  user  selects  an  acceptable  level  of  numerical  truncation  error 
(per  step)  in  the  state  of  the  object,  and  the  method  then  estimates  and  controls  the  error  by  adjusting  the 
step  size  so  that  the  error  is  close  to  but  does  not  exceed  the  specified  error  tolerance.  As  a  result,  the  work 
devoted  to  each  step  and  the  accuracy  achieved  in  the  step  are  balanced  for  overall  efficiency  (Butcher  [44] 
and  Shampine  [237,  238]).  Note  that  fixed-step  propagators  do  not  reject  steps  because  there  is  no  accuracy 
requirement. 

For  an  object  in  highly-elliptic  orbit,  a  variable-step  propagator  will  take  smaller  steps  near  perigee  and 
larger  steps  near  apogee,  thereby  solving  the  IVP  in  an  efficient  manner.  Conversely,  fixed-step  propagators 
take  larger  steps  near  perigee  and  smaller  steps  near  apogee.  As  a  result,  they  are  forced  to  take  extremely 
small  time  steps  throughout  the  entire  propagation  in  order  to  achieve  the  same  accuracy  as  that  of  a  variable- 
step  method  of  the  same  order.  For  this  reason,  the  equations  of  motion  are  typically  transformed  (e.g., 
using  the  Sundman  or  generalized  Sundman  transform  [31])  so  that  fixed  steps  can  be  taken  in  an  orbital 
anomaly  (e.g.,  eccentric  anomaly  or  true  anomaly),  rather  than  in  time,  in  an  effort  to  distribute  the  numerical 
truncation  error  evenly  across  the  steps. 

As  previously  mentioned,  precise  uncertainty  propagation,  in  addition  to  precise  orbit  propagation,  is 
needed  to  support  numerous  methods  for  advanced  SSA.  A  large  class  of  methods  for  propagating  the  un¬ 
certainty  in  an  object’s  state  (i.e.,  its  probability  density  function)  requires  finding  high-order  numerical 
solutions  to  ensembles  of  IVPs.  Uncertainty  propagation  via  the  unscented  Kalman  filter  (UKF)  [156], 
for  example,  requires  2n  +  1  trajectory  (orbit)  propagations,  where  n  is  the  dimension  of  the  stated  The 
variable-step  implementation  of  GL-IRK  developed  by  Aristoff,  Florwood,  and  Poore  [11,  14,  15]  exploits 
the  proximity  of  the  particles  or  states  and  enables  them  to  be  propagated  collectively,  rather  than  individu¬ 
ally.  Specifically,  once  the  first  state  in  the  ensemble  is  propagated,  the  remaining  states  can  be  propagated 
at  a  fraction  of  the  cost  by  reusing  the  (near)  optimal  step  sizes  taken  during  propagation  of  the  first  state, 
and  by  using  the  solution  to  the  first  IVP  itself  to  warm-start  the  iterations  for  the  Runge-Kutta  equations 
arising  when  solving  the  remaining  IVPs.  For  the  perturbed  two-body  problem  of  orbital  mechanics,  this 
approach  was  found  to  significantly  reduce  the  computational  cost  of  uncertainty  propagation,  measured  by 
the  number  of  force-model  evaluations  for  a  given  accuracy  [11,  13,  14]. 

7.16  Recommendations 

Many  of  the  methods  for  uncertainty  propagation  are  surveyed  above.  Here  are  some  final  comments  and 
recommendations . 

1.  While  one  cannot  compare  all  the  methods  presented  above,  the  computational  results  in  Section  7.10 
^The  2n  +  1  states  arising  within  the  UKF  are  often  referred  to  as  sigma  points. 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


95 


7.16.  RECOMMENDATIONS 


CHAPTER  7.  PROPAGATION  OF  UNCERTAINTY 


indicate  the  performance  advantage  of  the  representation  of  uncertainty  in  orbital  element  space  and 
the  performance  of  the  UKF,  GVM  filter,  and  Gaussian  sums  in  element  space.  A  more  detailed  study 
that  uses  a  common  set  of  metrics  and  test  cases,  such  as  those  described  in  Section  9.1,  and  applies 
them  to  the  uncertainty  propagation  methods  reviewed  in  this  chapter  should  be  undertaken  in  the 
future. 

2.  The  above  outline  of  methods  currently  being  researched  by  the  astrodynamics  community  demon¬ 
strates  that  virtually  every  nonlinear  filtering  method  that  could  be  investigated  is  being  investigated 
to  propagate  uncertainties.  We  find  this  maturing  area  of  research  to  be  of  very  high  quality  with 
each  method  having  its  own  merits.  To  better  understand  the  strengths  (and  weaknesses)  of  the  above 
methods,  one  needs  a  set  of  test  problems,  a  set  of  performance  metrics,  and  a  list  of  test  cases  that  can 
help  evaluate  these  strengths  and  weaknesses.  Given  the  performance  metrics  discussed  in  Chapter  8 
and  the  test  cases  outlined  in  Chapter  9,  here  is  a  list  of  potential  comparisons: 

•  an  assessment  of  the  range  of  validity  for  each  method  for  propagating  uncertainty  as  a  function 
of  the  initial  uncertainty, 

•  performance  with  and  without  process  noise  or  stochastic  acceleration, 

•  performance  with  and  without  model  mismatch, 

•  computational  complexity  and  runtime, 

•  performance  in  different  regimes  of  space. 

In  addition  to  propagation  of  uncertainty  forward  in  time,  the  transformation  of  uncertainty  from  one 
coordinate  system  to  another  should  be  considered. 

3.  With  respect  to  orbital  propagation,  the  use  of  one  that  can  control  the  error  relative  to  the  model 
dynamics  is  highly  recommended  so  that  the  error  in  the  propagated  state  is  guaranteed  to  be  sub¬ 
stantially  below  the  uncertainties  in  the  dynamics  including  initial  conditions.  The  newer  orbital 
propagators  based  on  implicit  Runge-Kutta  methods  or  modified  Picard  iteration  as  presented  in  Sec¬ 
tion  7.15  should  be  considered  due  to  their  numerical  stability  properties,  efficiency  due  to  large  step 
sizes,  adaptability  to  high  performance  computing  and  parallelization,  and  the  ability  to  propagate  an 
ensemble  of  orbits  needed  in  the  unscented  Kalman  filter,  Gauss-Hermite  filters,  Gauss  von  Mises 
filter,  or  particle  filters. 

4.  While  the  methods  of  this  chapter  address  covariance  and  uncertainty  realism  during  propagation,  the 
sensitivity  of  each  of  the  four  mission  areas  presented  in  Chapter  3  to  degraded  covariance/uncertainty 
realism  should  be  investigated  in  conjunction  with  the  above  metrics.  Such  an  understanding  might 
help  set  some  of  the  metric  parameters  and  be  useful  in  decision  making  processes. 
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8.1  Introduction 

This  chapter  addresses  metrics  for  covariance  and  uncertainty  realism  for  aleatoric  uncertainties  in  the  prop¬ 
agated  state  of  an  object.  Central  to  this  type  of  uncertainty  are  the  properties  of  a  point  estimator  which 
uses  sample  data  to  calculate  a  single  value  or  statistic.  Two  desirable  properties  of  a  point  estimator  are  that 
it  be  unbiased  and  consistent.  The  estimator  is  unbiased  if  the  difference  between  the  expected  value  and 
the  true  value  of  the  parameter  being  estimated  is  zero.  It  is  consistent  if  the  expected  value  of  the  estimator 
converges  in  probability  to  the  true  value  of  the  parameter  being  estimated  as  the  sample  size  increases.  Un¬ 
der  Gaussian  assumptions,  covariance  realism*  is  the  proper  characterization  of  the  covariance  (statistical 
uncertainty)  in  the  state  of  a  system  -  that  is,  the  rigorous  estimation  of  both  its  mean  and  covariance.  Co- 
variance  realism  thus  requires  that  the  estimate  of  the  mean  be  the  true  mean  (i.e.,  the  estimate  is  unbiased) 
and  the  covariance  possesses  the  right  size,  shape,  and  orientation  (i.e.,  consistency).  Relaxing  Gaussian  as¬ 
sumptions,  uncertainty  realism  is  the  proper  characterization  of  the  uncertainty  in  that  state  using  a  general 
(i.e.,  non-Gaussian)  probability  density  function.  Uncertainty  realism  requires  that  all  cumulants  (beyond 
the  state  and  covariance)  be  properly  characterized.  The  relationship  between  covariance  realism  and  un¬ 
certainty  realism  is  that  the  former  is  the  necessary  but  not  the  sufficient  condition  for  achieving  the  latter. 
The  two  definitions  coincide  if  the  process  is  Gaussian.  In  future  systems  wherein  the  number  of  objects 
may  increase  tenfold,  the  number  of  sensor  reports  for  each  space  object  may  necessarily  decrease,  even  for 
objects  in  the  space  catalog,  nonlinearity  may  have  an  increasing  impact  on  the  uncertainty.  Also,  for  UCTs, 
which  generally  have  larger  covariances  than  those  in  the  space  catalog  or  for  cases  where  longer-term  pre¬ 
diction  is  needed,  such  as  for  conjunction  assessments,  non-Gaussian  representations  of  uncertainty,  such 
as  those  described  in  Chapter  7,  may  be  necessary  to  characterize  the  actual  uncertainty  in  the  state  of  the 
space  objects.  We  thus  consider  metrics  for  assessing  both  covariance  realism  and  uncertainty  realism. 

Metrics  for  evaluating  covariance  realism  have  been  in  use  in  astrodynamics  community  for  some  time. 
These  can  be  found  in  the  paper  by  Vallado  and  Seago  [265],  references  therein,  and  the  report  [105]. 
One  of  these  metrics  is  based  on  the  averaged  Mahalanobis  distance  metric  and  has  been  used  by  several 
groups  [16,  68,  115,  231]  in  assessing  covariance  realism.  Specifically,  this  metric  assesses,  for  a  particular 

*The  term  covariance  consistency  [76]  is  used  in  other  tracking  domains  in  place  of  the  term  “covariance  realism.” 
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object,  the  mean  normalized  chi-squared  statistic  of  the  track  (orbit)  assigned  to  that  object.  This  metric 
is  defined  using  the  square  of  the  Mahalanobis  distance  [184]  and  is  necessary  for  covariance  realism,  but 
is  not  sufficient.  More  robust  and  general  metrics  are  needed  for  both  covariance  realism  and  uncertainty 
realism.  Specifically,  the  following  attributes  of  a  metric  are  desirable. 

1 .  Metrics  should  be  computationally  tractable  and  statistically  based. 

2.  Metrics  should  be  sufficiently  powerful  to  detect  breakdown  of  covariance  and  more  generally  uncer¬ 
tainty  realism. 

3.  Metrics  should  be  compatible  with  any  choice  of  coordinate  system  used  to  represent  uncertainty  (i.e., 
they  should  not  be  restricted  to  Cartesian  representations  of  the  covariance  matrix  or  more  generally 
uncertainty). 

4.  Metrics  should  address  both  covariance  and  uncertainty  realism,  i.e.,  non-Gaussian  realism. 

5.  Metrics  should  be  non-intrusive  to  avoid  the  need  to  share  proprietary  algorithms,  models,  and  soft¬ 
ware. 

To  achieve  these  objectives,  we  present  and  discuss  several  potential  metrics.  In  order  to  propose  met¬ 
rics  that  enable  the  assessment  of  uncertainty  realism  for  more  general  probability  density  functions,  e.g.,  a 
Gaussian  sum,  it  is  necessary  to  generalize  the  Mahalanobis  distance  to  one  that  that  tests  the  proper  charac¬ 
terization  of  the  probability  density  function  of  the  state,  i.e.,  uncertainty  realism.  Under  weak  assumptions 
as  explained  in  Section  8.3,  this  new  uncertainty  realism  metric  is  also  a  chi-squared  random  variable.  As 
such,  analogous  tests  for  covariance  realism  (both  in  off-line  simulations  with  multiple  Monte-Carlo  trials 
and  online  with  real  data)  can  be  extended  to  use  the  uncertainty  realism  metric,  some  of  which  are  reviewed 
in  this  report,  such  as  Pearson’s  chi-squared  goodness-of-fit  test  [247]  and  the  Cramer-von  Mises  goodness- 
of-fit  test  [59,  78,  250].  We  acknowledge  that  the  uncertainty  realism  metric  and  the  corresponding  tests 
for  uncertainty  realism  proposed  in  this  report  only  address  aleatoric  uncertainties  (which  tit  naturally  in  a 
probabilistic  framework)  and  not  epistemic  uncertainties  (which  are  generally  treated  by  non-probabilistic 
methods). 

The  remainder  of  this  chapter  is  as  follows.  The  classical  Mahalanobis  distance  and  its  use  for  testing 
covariance  realism  is  reviewed  in  Section  8.2.  Since  distributions  other  than  Gaussians  arise,  we  then  discuss 
how  the  Mahalanobis  distance  can  be  generalized  to  non-Gaussian  probability  density  functions  (PDFs)  in 
Section  8.3,  thereby  defining  a  metric  for  uncertainty  realism.  An  averaged  uncertainty  realism  metric  is 
then  presented  in  Section  8.4.  The  more  powerful  distribution  matching  tests  and  examples  are  presented  in 
Section  8.5  and  a  summary  of  the  recommendations  in  Section  8.6. 

8.2  Mahalanobis  Distance 

The  track  (orbit)  covariance  realism  metric  evaluates  the  consistency  of  the  uncertainty  corresponding  to 
the  orbit  state  estimate  against  some  known  truth  state.  The  uncertainty  in  the  state  of  an  object  is  often 
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represented  by  a  Gaussian.  Recall  that  the  random  vector  x  £  Mn  is  jointly  distributed  as  a  Gaussian 
distribution  if  and  only  if  its  joint  PDF  has  the  form 


px(x;n,  P)  =  Af(x\n,  P) 


1 

y/det(27rP) 


exp 


/i)TP_1(* 


(8.2.1) 


In  this  definition,  p  £  Mn  denotes  the  mean  (which  also  coincides  with  the  mode)  and  P  is  an  n  x  n 
symmetric  positive-definite  matrix  called  the  covariance.  Under  these  Gaussian  assumptions,  the  covariance 
realism  metric  is  defined  using  the  square  of  the  Mahalanobis  distance.  Let  x  be  a  given  (Gaussian)  orbital 
state  estimate  at  a  certain  time  t  and  let  P  be  its  corresponding  estimated  covariance.  Further  denote  Xtruth 
as  the  truth  state  of  the  target  at  time  t.  The  squared  Mahalanobis  distance  between  the  estimated  orbit  state 
and  truth  target  state  is  defined  as 


AA  (x ,  Xtruth .)  P )  (x  35trut/i)  -f*  (®  ® truth )•  (8.2.2) 

The  expected  value  of  AA  is  n,  where  n  is  the  dimension  of  the  state  vector  x.  Moreover,  it  follows  that 
AA  ~  \'2(n);  i.e.,  AA  is  chi-squared  distributed  with  n  degrees  of  freedom. 

Given  a  significance  level  a  (typically  0.01  or  0.001),  one  can  derive  a  two-sided  100(1 — ck)%  confidence 
interval  for  the  distribution  \'2(n)/?r  given  by  [\2(n;  a/2)/n,  x2(n'i  1  ~  ck/2 ) /r?.] ,  where  x2(n;/3)  is  the 
100/3%  quantile  of  the  distribution  x'2(n)-  One  rejects  the  null  hypothesis  that  the  covariance  P  centered  at 
the  estimate  x  is  realistic  given  the  truth  xtruth  if  AA{x\  xtruth,  P) /n  falls  outside  this  two-sided  confidence 
interval.  For  example,  if  a  =  0.001  and  n  =  6,  the  two-sided  99.9%  confidence  interval  for  this  test  is 
[0.0499,4.017], 

We  remark  that  (8.2.2)  can  be  modified  to  accommodate  the  case  in  which  the  truth  state  xtruth  is 
imprecise  or  “fuzzy.”  If  the  uncertainty  in  xtruth  is  provided  as  a  covariance  matrix  P truth  then,  in  place  of 
(8.2.2),  one  can  use 


AA.  {xi  aJtrut/i;  Ptrut/ii  P)  —  (®  Xtruth)  (P  H“  Htruth)  {x  Xtruth) .  (8.2.3) 

The  use  of  a  covariance  for  the  truth  is  discussed  more  generally  in  the  report  by  Frisbee  [155]. 

The  covariance  realism  metric  based  on  the  squared  Mahalanobis  distance  (8.2.2)  or  (8.2.3)  has  many 
applications  both  at  the  sensor-  and  system-level.  One  example  of  its  use  for  validating  covariance  realism 
during  uncertainty  propagation  is  the  following.  First,  we  make  the  following  basic  assumptions:  (i)  the 
orbital  state  uncertainty  is  identically  Gaussian  and  is  realistic^  at  some  initial  epoch  to,  and  (ii)  the  truth 
state  Xtruth  is  available  at  some  future  time  t  >  to.  With  these  assumptions,  we  propagate  the  Gaussian  at 
epoch  to  to  time  t,  approximate  the  propagated  uncertainty  by  a  Gaussian  with  state  (mean)  x  and  covariance 
P,  and  evaluate  the  metric  (8.2.2).  The  application  of  this  metric  is  only  valid  if  the  propagated  uncertainty 
is  represented  by  a  Gaussian  distribution.  A  generalization  of  the  covariance  realism  metric  to  an  uncertainty 
realism  metric  that  relaxes  the  assumptions  that  the  initial  or  propagated  uncertainty  be  Gaussian  is  described 
next. 

1  In  this  example,  we  will  not  be  concerned  about  how  this  initial  Gaussian  covariance  is  generated  and  if  it  is  indeed  realistic. 
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8.3  Generalization  of  the  Mahalanobis  Distance 


In  order  to  treat  probability  density  functions  other  than  a  Gaussian,  it  is  necessary  to  generalize  the  Maha¬ 
lanobis  distance.  Let  px(x;  0)  denote  a  general  PDF  in  the  //-dimensional  orbit  state  x  with  some  parameter 
set  0.  The  proposed  generalization  of  the  squared  Mahalanobis  distance  which  serves  as  a  metric  for  un¬ 
certainty  realism  is  defined  by 


U.(x\  0)  =  —2  In 


Px(x-,@Y 
Px(x;@)\  ’ 


(8.3.1) 


where  x  is  the  mode  of  x\ 


x  =  argma  \xpx(x;Q). 


We  remark  that  for  a  Gaussian  PDF,  as  defined  in  (8.2.1),  the  parameter  set  0  encapsulates  the  mean  pi  and 
covariance  P.  Further,  in  such  a  case,  the  uncertainty  realism  metric  (8.3.1)  reduces  to 


U{x\  0)  =  U(x]  pi,  P)  =  {x  —  pi)T P  l(x  —  pi), 


which  is  precisely  the  squared  Mahalanobis  distance  (8.2.2)  defined  earlier.  The  uncertainty  realism  metric 
(8.3.1)  is  chi-squared  distributed  or  approximately  so  under  the  following  scenarios. 

Suppose  again  that  px(x;@)  =  J\f(x;  pi,P),  and  let  x  =  <f>(y)  where  $  is  a  volume -preserving 
transformation*  (i.e.,  the  determinant  of  its  Jacobian  is  unity).  By  the  change  of  variables  theorem,  it  fol¬ 
lows  that  py(y;  0)  =  A pi,  P)  and,  from  the  definition  of  the  uncertainty  realism  metric  (8.3.1), 
U(y;&)  =  pi,  P).  Thus,  quite  remarkably,  the  uncertainty  realism  metric  for  any  (possibly 

non-linear)  volume -preserving  transformation  of  a  Gaussian  random  vector  also  possesses  the  chi-squared 
property. 

The  property  presented  above  is  significant  with  regards  to  uncertainty  propagation  of  a  space  object’s 
orbital  state  under  perturbed  two-body  dynamics  and  no  process  noise.  Since  the  dynamics  are  dominated 
by  conservative  forces  (i.e.,  gravity),  any  initial  Gaussian  distribution  that  is  propagated  under  said  dynam¬ 
ics  will  have  the  form  det(d& /dy)  AT (&(y);  pi,  P),  where  det (d&/dy)  ~  1.  This  is  a  consequence  of 
Liouville’s  theorem  [101]  in  Hamiltonian  mechanics.  Thus,  the  uncertainty  realism  metric  (8.3.1)  applied 
to  this  distribution  will  be  (approximately)  chi-squared  distributed. 

Another  example  is  that  of  Gaussian  mixtures,  which  are  used  by  many  authors  [67,  122,  125,  138,  255] 
to  represent  a  general  probability  density  function.  If  the  Gaussian  mixture  well  approximates^  the  actual 
distribution  det(d$>/dy)  AT(&(y);  pi,  P),  then  one  can  apply  the  definition  (8.3.1)  directly  to  the  Gaussian 
mixture  knowing  that  the  resulting  test  statistic  is  (approximately)  chi-squared  distributed. 

Finally,  a  Gauss  von  Mises  distribution  [123,  126,  127]  is  a  new  distribution  for  representing  uncertainty 
in  orbital  element  space.  It  has  been  shown  to  have  superior  properties  under  nonlinear  uncertainty  propaga¬ 
tion  compared  to  a  Gaussian  when  propagated  using  the  (prediction  steps  of)  the  UKF  or  EKF.  Under  weak 
conditions,  U[x\  0)  is  also  chi-squared  distributed  with  n  degrees  of  freedom  [127].  Thus,  in  analogy  to 

*One  example  of  a  volume-preserving  transformation  is  the  solution  flow  of  a  conservative  dynamical  system  (without  process 
noise)  that  propagates  some  initial  state  xo  at  time  to  to  a  state  x  at  time  t . 

Tn  principle,  any  PDF  can  be  approximated  by  a  Gaussian  mixture  to  within  any  desired  accuracy  (in  the  L 1  sense)  due  to  a 
result  of  Alspach  and  Sorenson  [7], 
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Figure  8.3.1  Setup  for  the  statistical  significance  tests.  The  yellow  shaded  regions  are  those  regions  in  which 
the  null  hypothesis  is  rejected.  Dependence  on  the  parameter  set  0  in  f(x\  0)  is  omitted  in  the  figure. 


the  statistical  test  for  covariance  realism  described  at  the  end  of  Subsection  8.2,  the  metric  (8.3.1)  provides 
a  test  for  uncertainty  realism. 

8.3.1  Statistical  Interpretation 

The  statistical  interpretation  of  the  Mahalanobis  distance  metric  (8.2.2)  and  its  generalization,  given  by 
(8.3.1),  can  be  understood  by  considering  the  general  setting  of  a  multivariate  random  vector  x  with  support 
on  a  differentiable  manifold  SQt.  We  express  its  PDF  in  the  form  px(x;  0)  =  e~Ex''&)/2  f(x;  0)  = 
—2  In px(x;  0).  Suppose  now  that  a  point  x*  £  SOI  is  given  and  one  wishes  to  test  the  null  hypothesis  Hq 
that  x*  is  not  a  statistically  significant  realization  of  the  random  vector  x  (i.e.,  cc*  is  a  representative  draw 
from  x).  The  p- value  for  a  one-sided  test  is 

p  =  Pr[a:  £  0*]  =  f  e~^x’&^2  dx,  (8.3.2) 

Jn, 

where  0*  =  {x  \  f(x;  0)  >  /(**;  0)  =  C}.  Smaller  p- values  imply  that  the  realization  a*  lies  farther 
out  on  the  tails  of  the  PDF  (see  Figure  8.3.1(a)).  The  null  hypothesis  Hq  is  rejected  at  the  significance  level 
a  (e.g.,  0.01)  if  p  <  a.  Figure  8.3.1(b)  shows  the  setup  for  the  analogous  two-sided  hypothesis  test.  For  a 
given  significance  level  a,  one  determines  the  contours  Cl  and  Cu  such  that 

[  e-/(*;e)/2da;=  f  e-/(*;e)/2dx=l 

where  Ql  =  {x  \  f(x;  0)  <  Cl}  and  Djj  =  {x  \  f(x\  0)  >  Cu}.  (Note  that  the  yellow  shaded  region  in 
Figure  8.3.1(b)  has  probability  a.)  A  two-sided  test  with  significance  level  a  rejects  the  null  hypothesis  Ho 
if  /(a:*;  0)  <  CL  or  /( cc*;  0)  >  Cjj. 

8.3.2  Examples 

Suppose  px(x ;  0)  =  J\f(x-,  p,  P)  (i.e.,  x  is  Gaussian)  and  a  realization  x *  £  Mn  is  given,  then 
f(x;  0)  =  — 21nA f{x\ p,  P)  =  M(x]  p,,  P)  +  lndet(27rP), 
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where  A4  is  the  Mahalanobis  distance  (8.2.2).  The  integration  region  (2*  in  (8.3.2)  is 


Substituting  this  information  into  (8.3.2)  yields 


p  =  Pr[a:  G  0*]  =  Pr [M(x;  p,  P)  >  M(x*;  p,,  P)]  =  Pr[x2(n)  >  M(x*-,n,  P)]. 


Thus,  the  resulting  p- value  can  be  computed  by  evaluating  the  complementary  cumulative  distribution  func¬ 
tion  (i.e.,  tail  distribution)  of  %2(n)  at  the  Mahalanobis  distance  (8.2.2)  evaluated  at  the  realization  x *. 

The  uncertainty  realism  metric  (8.3.1)  can  also  be  applied  in  analogy  to  the  example  discussed  in  Sub¬ 
section  8.2  on  uncertainty  propagation.  We  make  the  analogous  basic  assumptions  that  (i)  the  orbital  state 


uncertainty  is  represented  by  some  general  PDF  (that  need  not  be  Gaussian)  and  is  realistic  at  some  initial 


epoch  to,  and  (ii)  the  truth  state  xtruth  is  available  at  some  future  time  t  >  to-  With  these  assumptions,  we 
propagate  the  initial  PDF  at  epoch  to  to  time  t  yielding  the  PDF  px(x;  0),  and  evaluate  the  uncertainty  re¬ 
alism  metric  lA{xtruth\  ©)  from  the  definition  (8.3.1).  Based  on  some  confidence  interval  of  the  underlying 
chi-squared  distribution,  we  might  reject  the  null  hypothesis  that  the  propagated  PDF  “captures  the  truth”; 
i.e.,  Xtruth  is  a  representative  draw  from  px(x ;  ©). 

8.3.3  Important  Remarks 

It  is  important  to  understand  that  covariance  realism  does  not  always  imply  uncertainty  realism.  Said  in 
another  way,  if  one  has  properly  characterized  the  first  two  cumulants  (i.e.,  mean  and  covariance)  of  the 
state  PDF,  then  it  does  not  necessarily  imply  that  those  two  cumulants  alone  are  sufficient  for  characterizing 
the  full  PDF.  Consequently,  while  our  proposed  tests  for  uncertainty  realism  can  detect  a  breakdown  in 
uncertainty  realism,  such  a  breakdown  does  not  say  anything  about  the  realism  of  the  covariance^.  This  is 
a  moot  point;  one  needs  to  strive  for  uncertainty  realism.  Henceforth,  we  will  use  the  terms  “uncertainty 
realism  metric”  and  “uncertainty  realism  test”  exclusively. 

8.4  Averaged  Uncertainty  Realism  Metric 

As  an  uncertainty  realism  metric,  one  can  consider  the  value  of  U  (or  A4  in  the  Gaussian  case),  averaged  over 
all  orbits  or  at  each  time  instance  (averaged  over  one  or  many  Monte-Carlo  trials,  or  with  real  data),  together 
with  upper  and  lower  bounds  for  a  particular  confidence  interval.  Specifically,  let  be  the  uncertainty 
realism  metric  (8.3.1)  computed  in  the  i-th  Monte-Carlo  trial.  Let  k  be  the  total  number  of  independent 


trials.  Then, 


(8.4.1) 


^If  one  is  only  interested  in  testing  covariance  realism  without  regard  to  uncertainty  realism,  some  options  are  provided  by 
Vallado  and  Seago  [265]. 
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One  can  test  the  null  hypothesis  that  the  (normalized)  average  of  the  samples  given  by  U.  is  consistent 
with  the  mean  of  the  distribution  x2  (nk)  /  (nk) .  For  example,  if  n  =  6  and  k  =  100,  a  two-sided  99.9% 
confidence  interval  for  the  distribution  (8.4.1)  is  [0.8209, 1.2010].  Note  that  this  is  a  necessary  (but  not 
sufficient)  test  of  uncertainty  realism,  and  that  E [U\  =  1.  Moreover,  Var[Z7]  — >  0,  as  k  — >  oo  (i.e.,  the 
confidence  interval  becomes  infinitesimally  small). 

As  motivated  in  Section  8.5,  a  stronger  test  for  uncertainty  realism  (in  an  off-line  setting  with  multiple 
Monte-Carlo  trials)  would  be  to  consider  the  distribution  of  the  Monte-Carlo  samples  UN>  and  perform  a 
distribution  matching  or  goodness-of-fit  test  [247].  This  test  would  indirectly  consider  the  consistency  of 
the  sample  with  both  the  mean  and  higher-order  cumulants  of  the  matching  chi-squared  distribution.  The 
example  below  highlights  some  of  the  differences  between  the  averaged  uncertainty  realism  metric  and 
metrics  based  on  distribution  matching  and  motivates  the  need  for  the  latter  more  powerful  tests. 

8.4.1  Averaged  Uncertainty  Realism  Metric  for  the  Propagation  of  Uncertainty 

An  off-line  simulation  with  multiple  Monte-Carlo  trials  that  applies  the  metric  (8.4.1)  for  assessing  uncer¬ 
tainty  realism  during  uncertainty  propagation  is  the  following.  First,  we  make  the  basic  assumption  that  the 
orbital  state  uncertainty  is  identically  Gaussian  and  is  realistic  at  some  initial  epoch  to-  With  this  assumption, 
we  perform  the  following  operations: 

1 .  Propagate  the  Gaussian  at  epoch  t,Q  to  time  t  and  approximate  the  propagated  uncertainty  by  a  Gaus¬ 
sian  with  state  (mean)  //  and  covariance  P. 

2.  Sample  random  particle  states!!  x-1'1  (to),  i  =  1, . . . ,  k,  from  the  initial  Gaussian  distribution  at  epoch 
to- 

3.  Propagate  each  particle  state  (f0  j  from  time  to  to  time  t  yielding  a  propagated  particle  state  (ty 

4.  For  i  =  1, . . . ,  k,  compute  U®  =  A4  (®W(f);  /u,  P). 

5.  Compute  the  averaged  uncertainty  realism  metric  U  from  (8.4.1)  in  conjunction  with  the  metrics  U{k 
computed  in  the  previous  step* **. 

6.  Perform  the  hypothesis  test  described  in  the  text  following  Equation  (8.4.1). 

Figure  8.4.1  illustrates  the  differences  between  the  conclusions  one  might  make  when  applying  the 
averaged  uncertainty  realism  metric  described  here  and  a  distribution  matching  test.  If  one  were  to  plot  a 
histogram  of  the  Monte-Carlo  samples  U^\  it  could  look  like  the  one  shown  in  the  figure.  In  this  example, 
computing  the  average  of  the  would  yield  a  value  of  n,  in  agreement  with  the  expectation  of  the  \'2(n) 
distribution.  Consequently,  one  would  be  tempted  to  assert  that  one  has  uncertainty  realism  since  the  test 
statistic  (i.e.,  U)  would  lie  in  the  center  of  any  confidence  interval.  Indeed,  consistency  of  the  sample  mean 

^  A  random  draw  x  from  a  multivariate  Gaussian  with  mean  fj,  and  covariance  P  can  be  obtained  as  follows.  Let  z  be  a  vector 
where  each  component  is  an  independent  random  draw  from  the  standardized  Gaussian  (i.e.,  the  Gaussian  with  mean  0  and  variance 
1);  this  functionality  is  provided  in  most  programming  languages  and  scientific  and  statistical  software.  Then,  the  required  x  is  the 
vector  (j,  +  Az,  where  A  is  the  lower-triangular  Cholesky  factor  [102]  of  the  covariance  P  such  that  P  =  AAr. 

**  Alternatively  or  in  addition  to  Steps  5  and  6,  one  can  compute  the  normalized  Pearson  test  statistic  or  the  Cramer- von  Mises 
test  statistic  from  the  and  perform  the  corresponding  hypothesis  tests  described  in  Subsection  8.5. 
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Figure  8.4.1  Depiction  of  a  distribution  matching  test.  Shown  is  a  histogram  of  Monte-Carlo  trials  with 
the  expected  x2(n)  matching  distribution.  In  this  example,  the  sample  mean  (e.g.,  as  would  be  computed 
from  the  uncertainty  realism  test  described  in  Subsection  8.4)  is  consistent  with  the  expected  value  of  the 
matching  distribution;  the  sample  variance  (and  higher-order  moments)  would  not  be  consistent.  Thus,  a 
distribution  matching  test  would  correctly  reject  the  null  hypothesis  that  the  samples  come  from  a  \'2(n) 
distribution;  a  first-order  moment  (mean)  matching  test  would  not. 


with  the  expected  value  of  the  matching  distribution  is  only  a  necessary  condition  for  uncertainty  realism. 
Clearly,  as  the  figure  shows,  the  variance  (as  well  as  the  higher-order  moments)  of  the  U^1'  do  not  match 
those  of  the  target  chi-squared  distribution;  the  histogram  is  a  poor  fit.  Therefore,  a  distribution  matching 
test,  such  as  one  based  on  Pearson’s  test  or  the  Cramer-von  Mises  test  described  in  the  next  subsection,  is  a 
better  test  for  uncertainty  realism  in  this  case. 

8.4.2  Averaged  Uncertainty  Realism  Metric  and  Orbit  Determination 

The  above  sections  on  metrics  have  focused  on  the  application  to  the  propagation  of  a  Gaussian  (covariance 
realism)  and  a  general  probability  density  function  (uncertainty  realism).  These  metrics  are  often  used  in 
several  different  contexts. 

•  For  a  single  orbit  and  real  data  with  fuzzy  truth,  one  can  compute  the  (squared)  generalized  Maha- 
lanobis  distance  U W  in  Equation  (8.3. 1)  for  a  sequence  of  orbit  updates,  i.e.,  at  the  times  of  the  sensor 
report  updates,  and  then  apply  the  average,  generalized  Mahalanobis  distance  test  8.4.1. 

•  Again  for  a  single  orbit  but  in  a  simulation  environment,  we  can  use  multiple  Monte  Carlo  runs  with 
different  draws  of  the  sensor  reports  (at  the  same  times)  and  then  proceed  as  in  the  previous  example. 

•  Given  an  ensemble  of  orbits  and  truth,  one  can  compute  the  (squared)  generalized  Mahalanobis  dis¬ 
tance  in  Equation  (8.3.1)  and  used  the  generalized  Mahalanobis  8.4.1  for  the  ensemble. 

8.5  Distribution  Matching  Tests 

It  is  helpful  to  keep  in  mind  that  the  purpose  of  the  multivariate  PDF  is  to  represent  the  statistical  distribution 
of  the  actual  state  errors.  Durable  testing  of  the  adequacy  of  this  PDF,  therefore,  must  be  an  exercise  in 
distribution  matching.  Said  another  way,  it  must  be  a  determination  of  whether  the  statistical  distribution 
of  a  set  of  state  errors,  usually  calculated  by  the  comparison  of  state  estimates  to  an  externally-determined 
precision  reference  orbit,  matches  to  a  reasonable  degree  the  distribution  represented  by  the  hypothesized 
multivariate  PDF.  Testing  approaches  that  do  less  than  this,  such  as  distribution-matching  of  only  a  single 
component  of  the  state  error  or  the  testing  only  for  a  matching  of  distribution  mean  values  (as  in  the  averaged 
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uncertainty  realism  metric  described  in  Subsection  8.4),  render  results  that  can  be  of  some  use;  but  they  will 
not  allow  a  definitive  assessment  of  the  realism  of  the  uncertainty.  It  is  thus  important  to  discuss  fully- 
formed  methods  of  uncertainty  realism  and  the  particular  virtues  that  they  possess  so  that  the  advantages 
and  drawbacks  of  more  abbreviated  methods  can  be  thrown  into  relief. 

Most  multivariate  distribution  matching  approaches  proceed  by  calculating  test  statistics  (e.g.,  a  Ma- 
halanobis  distance)  that  can  then  be  evaluated  for  conformity  to  a  canonical  univariate  distribution  (e.g.,  a 
X2  distribution)  and  thus  employ  standard  goodness-of-fit  (GOF)  techniques.  GOF  approaches  comprise 
both  specialty  tests  for  a  specific  distribution  type  (a  number  of  these,  for  example,  exist  for  the  Gaussian 
distribution)  and  more  general  tests  that  are  capable  of  evaluating  conformity  to  a  number  of  different  dis¬ 
tributions.  Because  covariance  realism  evaluation  usually  requires  the  evaluation  of  conformity  to  both  the 
Gaussian  and  the  chi-squared  distribution,  and  because  uncertainty  realism  evaluation  can  involve  testing 
for  conformity  to  a  variety  of  different  distributions  (including  potentially  distributions  that  have  no  analytic 
representation),  one  should  focus  on  these  more  fungible  techniques,  principal  among  them  the  traditional 
chi-squared  distribution  matching  test  and  the  family  of  empirical  distribution  function  (EDF)  tests.  An  ex¬ 
ample  of  the  former,  Pearson’s  chi-squared  GOF  test,  is  discussed  in  Subsection  8.5.1,  while  examples  of  the 
latter,  including  the  Kolmogorov-Smirnov,  Cramer-von  Mises,  and  Anderson-Darling  tests,  are  described  in 
Subsection  8.5.2.  Other  possible  GOF  and  EDF  tests  and  our  reasons  for  not  using  them  to  assess  covariance 
and  uncertainty  realism  in  space  surveillance  applications  are  briefly  discussed  in  Subsection  8.5.3. 


8.5.1  Pearson’s  Chi-Squared  Goodness-of-Fit  Test 

Pearson's  chi-squared  distribution  matching  test  can  be  considered  the  most  ecumenical  in  that  it  can  be 
deployed  as  a  test  of  conformity  to  any  distribution  and  can  be  used  for  discrete  as  well  as  continuous 
distributions.  Further,  as  a  staple  of  most  introductory  statistics  courses,  it  has  the  additional  advantage  of 
familiarity  to  most  scientists  and  engineers.  When  testing  a  random  sample  to  see  if  it  can  be  considered 
to  belong  to  a  hypothesized  parent  distribution,  the  basic  procedure  is  to  (i)  divide  up  the  possible  range  of 
values  of  the  random  sample  into  a  set  of  m  discrete  cells;  (ii)  determine  the  number  of  values  that  actually 
fall  into  each  cell  and  the  number  that  should  have  fallen  into  each  under  the  assumed  parent  distribution; 
and  (iii)  compute  the  chi-squared  test  statistic  (defined  below)  and  compare  it  to  a  critical  value  from  the  \ ;2 
distribution.  It  what  follows,  we  describe  these  steps  in  more  detail. 

Specifically,  let  iW,  i  =  1, . . . ,  k,  denote  the  observed  sample  trials.  We  wish  to  test  the  null  hy¬ 
pothesis  that  these  observations  belong  to  a  particular  matching  distribution.  Suppose  the  observations 
are  grouped  into  m  cells  or  bins  where  Oj  is  the  number  of  observations  contained  in  the  j- th  bin.  The 
normalized  Pearson  test  statistic  is  defined  by 


p  =  1  \°j  ~  ej) 

x  m  —  1  —  v  4—'  e,- 

F  3=1  3 


(8.5.1) 


where  e3  is  the  expected  number  of  observations  contained  in  j -th  bin  as  determined  from  the  definition 
of  the  j- th  bin  and  the  properties  of  the  matching  distribution.  The  normalized  Pearson  test  statistic  Px  in 
(8.5.1)  asymptotically  approaches  the  distribution  \2 (ra  —  1  —  p)/(m  —  1  —  p)  where  the  integer  p  is  the 
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number  of  co- variates  used  in  fitting  the  matching  distribution.  For  example,  if  the  matching  distribution 
is  fixed  (e.g.,  to  a  x2(6)  distribution),  then  p  =  0.  The  approximation  of  Px  by  a  chi-squared  distribution 
breaks  down  if  the  expected  frequencies  e3  are  too  low.  One  way  to  ensure  that  the  e3  are  sufficiently  large 
(so  that  the  asymptotic  property  of  the  Pearson  statistic  holds)  is  to  choose  m  “equiprobable”  bins  so  that 
e.j  =  k/m  for  all  j.  In  other  words,  the  bins  are  of  variable  width  all  having  the  same  expected  probabilities. 
One  prescription  for  choosing  the  number  of  bins  is 


m  =  max(5,  min(100, 0.01/c)). 


We  now  specialize  the  framework  of  this  goodness-of-fit  test  to  the  case  when  the  observed  samples  x<l> 
are  U W  coiTesponding  to  the  uncertainty  realism  metric  (8.3.1)  of  the  z-tli  trial.  (Recall  that  Subsection  8.4. 1 
provides  an  example  of  how  the  are  generated  in  an  uncertainty  propagation  scenario.)  In  both  cases, 
the  target  matching  distribution  is  y2(n),  where  n  is  the  dimension  of  the  orbital  state  space.  For  sake  of 
example,  we  take  n  =  6.  We  define  the  m  bins  so  that  the  expected  number  of  observations  ej  is  k/m. 
for  all  j.  Let  &W  6  {1, . . .  ,  m}  denote  the  bin  number  in  which  each  observation  belongs.  By  virtue 
of  the  choice  of  the  expected  frequencies  e3 .  these  bin  numbers  can  be  determined  from  the  cumulative 
distribution  function  (CDF)  of  the  matching  distribution.  For  a  x2(6)  matching  distribution,  its  CDF  enjoys 
a  particularly  simple  form: 


F(x) 


1  —  |e  x/2(x2  +  4x  +  8),  x  >  0, 

0,  otherwise. 


(8.5.2) 


It  follows  that 

frM  =  | mf(i^)], 

for  i  =  1, . . . ,  k.  The  number  of  observations  contained  in  each  bin  can  be  readily  determined  as  follows: 
(i)  initialize  Oj  =  0,  for  j  =  1 , ...  ,m,  and  (ii)  for  i  =  1, . . . , k,  do  o6(o  <—  ob(i )  +  1.  Finally,  we  compute 
the  normalized  Pearson  test  statistic: 

p  1  y{0j-e3f  _  1  ^(oj-k/m)*' 

x  m  —  1  '  e7-  m  —  1  k/m 

j= 1  J  3= 1 

Given  a  significance  level  a  (typically  0.01  or  0.001),  we  can  derive  a  one-sided  100(1  —  a)%  confidence 
interval  for  the  distribution  x2(m— l)/(m—  1)  given  by  [0,  x2im~  1;  1— a)/(m—  1)],  where  x2{m~  1;  P)  is 
the  100/3%  quantile  of  the  distribution  x2(m  —  1).  One  rejects  the  null  hypothesis  that  the  observed  samples 
sW  belong  to  a  x2(6)  distribution  if  the  normalized  Pearson  test  statistic  Px  computed  above  falls  outside 
this  confidence  interval. 


8.5.2  Empirical  Distribution  Function  (EDF)  Tests 

The  simplicity  and  flexibility  of  the  Pearson  GOF  test  make  it  quite  appealing,  but  it  unfortunately  harbors 
considerable  disadvantages  as  well.  The  first  is  a  relative  lack  of  power  when  used  to  test  against  continuous 
distributions.  Discretizing  a  continuous  distribution  by  dividing  it  up  into  individual  cells  eliminates  some  of 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


106 


8.5.  DISTRIBUTION  MATCHING  TESTS 


CHAPTER  8.  METRICS 


Figure  8.5.1  Depiction  of  the  Kolmogorov-Smirnov  test.  The  test  statistic,  up  to  a  normalization  in  the 
sample  size  k,  is  the  largest  deviation  between  the  hypothesized  CDF  F(x)  and  the  empirical  CDF  Ff.(x), 
as  indicated  by  the  black  arrow. 


[ 


the  distribution’s  information;  techniques  that  can  preserve  continuity  will  generally  render  more  powerful 
results.  The  second  is  an  inherent  arbitrariness  introduced  by  the  number  of  cells  selected.  Certain  studies 
have  shown  that  the  test  is  more  powerful  when  performed  with  cells  sized  so  as  to  be  equiprobable  (as 
detailed  in  Subsection  8.5.1),  and  cell  quantities  optimized  for  the  use  of  the  test  against  a  hypothesized 
Gaussian  distribution  can  be  recommended  [191].  Flowever,  there  is  no  set  of  such  recommendations  for 
the  general  case,  with  both  the  test  outcomes  and  the  resulting  statistical  inference  affected  by  different  bin 
quantities  and  sizes.  Given  these  problems,  it  is  typically  best  to  select  other  GOF  approaches  unless  one  is 
performing  tests  against  a  hypothesized  distribution  that  is  natively  discrete. 

The  leading  alternative  candidate  for  this  type  of  GOF  testing  is  the  family  of  tests  based  on  the  empir¬ 
ical  distribution  function  (EDF)  of  the  sample  data  and  the  hypothesized  distribution.  The  basic  idea  here 
is  to  (i)  calculate  the  differences  between  a  cumulative  distribution  function  (CDF)  for  the  hypothesized 
distribution  and  an  empirical  CDF  (ECDF)  for  the  sample  distribution;  (ii)  encapsulate  these  differences 
in  terms  of  a  GOF  test  statistic;  and  (iii)  determine  the  likelihood,  by  the  use  of  published  p-value  tables, 
that  the  sample  actually  could  have  the  hypothesized  distribution  as  a  parent.  Constructing  the  CDF  for  the 
hypothesized  distribution  and  the  ECDF  for  the  sample  distribution  is  straightforward  enough;  plotting  them 
on  the  same  graph  can  reveal  the  differences  visually.  At  this  point,  there  are  two  different  approaches  to 
determining  the  overall  amount  of  deviation.  The  supremum  approach  is  to  catalog  the  largest  deviation  be¬ 
tween  the  hypothesized  and  empirical  result  and  examine  the  p- value  associated  with  this  large  a  deviation; 
this  approach  is  the  basis  of  the  Kolmogorov-Smirnov  test. 

In  Figure  8.5.2,  the  hypothesized  distribution’s  CDF  F(x)  is  given  in  red  and  the  ECDF  F/.(.x)  for  the 
sample  is  shown  in  blue.  The  .x-axis  is  the  actual  function  value,  and  the  y-axis  is  the  cumulative  probability. 
The  largest  deviation  between  the  hypothesized  and  empirical  CDF  is  indicated  by  the  arrow.  In  applying 
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the  Kolmogorov-Smirnov  test,  one  computes  this  deviation  and  adjusts  it  for  the  sample  size  k: 


Dk  =  Vk  sup  | Fk(x)  -  F(x)\. 

X 


One  then  compares  the  Kolmogorov-Smirnov  test  statistic  Dk  to  a  table  of  p-values  in  order  to  determine 
the  likelihood  that  the  sample  set  could  have  originated  from  the  hypothesized  distribution.  The  formulation 
of  the  hypothesis  test  here  is  somewhat  unusual;  the  null  hypothesis  is  that  the  two  distributions  (hypothe¬ 
sized  and  sample)  are  the  same.  The  null  hypothesis  is  rejected  for  low  p-values.  This  approach  is  called 
“weak-hypothesis  testing”  and  is  occasionally  criticized  for  being  too  permissive.  Note  that  it  must  be  re¬ 
membered  that  the  purpose  here  is  not  to  identify  the  true  underlying  distribution,  only  to  determine  whether 
the  hypothesized  distribution  is  a  reasonable  candidate  for  a  parent  distribution  of  the  sample.  P-values  from 
0.1-1%  are  typically  used  as  the  rejection  threshold  in  GOF  testing.  It  is  the  authors’  experience  that  at  this 
significance  level,  mismatched  distributions  are  clearly  rejected,  even  with  small  sample  sizes. 

The  second  strain  of  EDF  GOF  test  approaches  are  the  quadratic  statistics.  These  approaches  sum  up  all 
of  the  deviations  and  use  this  sum  as  the  test  statistic.  The  canonical  summation  equation  is  the  following: 

/OO 

[Fk(x)  -  F(x)]  2tP(F(x))  d F{x).  (8.5.3) 

-OO 

The  weighting  factor  'i/j(F(x))  is  typically  either  set  to  unity  to  produce  the  Cramer-von  Mises  statistic  or 
to  a  function  that  will  give  more  weight  to  the  tails,  such  as  ^(Ffa))  =  l/[F{x){\  —  F(x))],  to  produce 
the  Anderson-Darling  statistic.  Testing  proceeds  in  the  same  way  as  that  described  for  the  Kolgomorov- 
Smirnov  test.  The  test  statistic  Qk  is  calculated,  and  tables  of  p-values  are  consulted  to  determine  the  signif¬ 
icance  level  for  the  test  statistic  indicated.  A  good  source  for  these  tables  is  the  monograph  of  D’Agostino 
and  Stephens  [59]  on  this  subject.  Tables  exist  for  all  of  the  major  distributions  (e.g.,  normal,  gamma, 
chi-squared,  von  Mises),  both  as  fully-specified  distributions  (“Case  0”)  and  as  distributions  in  which  distri¬ 
bution  parameters  are  represented  by  estimators  (“Cases  1-3”). 

With  regards  to  assessing  uncertainty  realism  using  an  EDF  test  in  the  context  of  the  example  of  Sub¬ 
section  8.4.1,  we  recommend  using  a  GOF  test  based  on  a  quadratic  statistic  rather  than  the  Kolmogorov- 
Smirnov  test.  The  latter  is  generally  considered  less  powerful  than  quadratic  tests  because  it  considers  only 
one  value  (i.e.,  the  largest  deviation).  Of  the  two  quadratic  tests  reviewed  here,  we  recommend  the  Cramer- 
von  Mises  test  over  the  Anderson-Darling  test  because  the  latter,  though  usually  considered  more  powerful, 
is  more  fragile  due  to  sensitivity  of  the  test  on  the  tails.  Specializing  (8.5.3)  to  t/>(F(x))  =  1,  the  resulting 
Cramer-von  Mises  test  statistic  is^ 


Qk 


i= 1 


(8.5.4) 


where  the  x^\  i  =  1, ...  ,k,  are  the  observed  samples  in  increasing  order. 

detailed  explanation  of  the  derivation  of  (8.5.4)  from  (8.5.3)  can  be  found  in  the  monograph  of  Darling  [60]. 
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8.5.3  Other  Distribution  Matching  Tests 

There  are  a  number  of  additional  GOF  testing  approaches  that  are  common  in  modern  engineering  practice 
that  were  not  pursued  here  for  reasons  of  both  suitability  and  convenience.  The  Akaike  Information  Crite¬ 
rion  [43]  is  an  entropy-based  approach  that  has  achieved  currency  for  model  adequacy  assessment.  While 
in  some  ways  it  is  more  powerful  than  traditional  hypothesis  testing,  its  chief  drawback  is  that  it  can  serve 
only  as  a  comparative  test  to  evaluate  the  relative  performance  of  two  or  more  models;  it  cannot  give  an 
absolute  evaluation,  in  a  p-value  sense,  of  the  conformity  of  the  data  to  any  single  hypothesized  distribu¬ 
tion.  Regression/correlation  GOF  tests,  such  as  the  well-known  Shapiro-Wilk  test  [239],  and  moment-based 
tests,  such  as  the  combination  third- fourth  moment  test  [58],  are  very  much  legitimate  GOF  tests  that  often 
can  confer  substantial  power.  However,  the  ability  to  deploy  them  requires  certain  a  priori  products:  for 
regression  tests,  it  is  a  set  of  data  weighting  coefficients  appropriate  to  the  hypothesized  distribution;  and 
for  moment-based  tests,  it  is  the  null  distributions  of  those  moments  for  the  hypothesized  distribution.  The 
present  authors  were  not  able  to  locate  published  sources  for  these  a  priori  products  for  the  chi-squared 
distribution,  which  is  one  of  the  principal  hypothesized  distributions  to  be  tested  for  the  present  application. 
While  it  may  be  possible  to  establish  these  a  priori  products  through  private  Monte-Carlo  studies,  this  was 
seen  as  unnecessary  labor  when  other  GOF  tests  -  equally  powerful  -  already  exist  with  the  deployment 
products  necessary  for  the  testing  of  all  of  the  hypothesized  distributions  presently  considered.  For  this  rea¬ 
son,  the  present  collection  of  GOF  test  was  limited  to  the  traditional  Pearson  test  and  the  mainstream  EDF 
tests  (Kolmogorov-Smirnov,  Cramer-von  Mises,  and  Anderson-Darling). 

8.5.4  An  Example  for  Orbit  Propagation 

We  now  provide  an  example  that  tests  uncertainty  realism  in  the  context  of  the  uncertainty  propagation 
scenario  described  in  Section  8.4.1  and  examines  the  power  and  effectiveness  of  the  averaged  uncertainty 
realism  test,  the  Pearson  goodness-of-fit  (GOF)  test,  and  the  Cramer-von  Mises  test.  The  high-level  setup 
for  these  tests  is  as  follows  [123].  The  initial  orbital  state  at  epoch  describes  a  high  accuracy  low  Earth  orbit 
(LEO)  object;  its  uncertainty  is  taken  to  be  Gaussian  in  the  osculating  equinoctial  orbital  element  coordinate 
system.  The  LEO  object  state  and  covariance  are  propagated  using  the  prediction  step  of  the  unscented 
Kalman  filter  [157]  (UKF);  individual  sigma  points  in  the  UKF  arc  propagated  using  an  implicit  Runge- 
Kutta-based  method  [12,  13]  in  conjunction  with  a  32  x  32  gravity  model  and  lunar-solar  perturbations.  A 
total  of  1000  Monte-Carlo  trials  are  used;  10  (equiprobable)  bins  are  defined  when  applying  the  Pearson 
test.  Three  variations  of  the  scenario  are  considered  which  differ  only  by  the  choice  of  coordinate  system 
used  to  the  represent  the  state  and  covariance.  These  coordinate  systems  are  (i)  osculating  equinoctial 
orbital  elements,  (ii)  Cartesian  position-velocity  coordinates,  and  (iii)  J2  equinoctial  orbital  elements  [10]. 
For  scenarios  that  use  the  latter  two  coordinate  systems,  the  initial  state  and  covariance  (defined  in  the 
osculating  elements  as  described  above)  are  converted  to  (and  approximated  by)  a  Gaussian  in  the  target 
coordinate  system  using  the  unscented  transform* . 

*The  initial  state  uncertainty  is  well-approximated  by  a  single  Gaussian  in  all  three  coordinate  systems,  as  evidenced  in  Fig¬ 
ure  8.5.4,  since  the  averaged  uncertainty  realism  metric,  Pearson  test  statistic,  and  Cramer-von  Mises  test  statistic  are  (approxi¬ 
mately)  equal  at  the  initial  epoch  and  well  within  their  respective  99.9%  confidence  intervals. 
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Figure  8.5.2  Application  of  the  (a)  averaged  uncertainty  realism  test,  (b)  Pearson  goodness-of-fit  test,  and  (c) 
Cramer-von  Mises  test  to  an  uncertainty  propagation  scenario  in  space  surveillance.  The  coordinate  system 
in  which  the  state  and  covariance  are  represented  is  Cartesian  position-velocity  coordinates  (red),  osculating 
equinoctial  orbital  elements  (green),  or  Jo  equinoctial  orbital  elements  (blue).  The  99.9%  confidence  region 
is  shown  in  the  shaded  area.  The  less-powerful  tests  in  (a)  and  (b)  suggest  that  uncertainty  realism  does  not 
break  down  in  osculating  element  space  (green)  over  the  entire  duration  of  propagation,  whereas  the  more- 
powerful  Cramer-von  Mises  test  (c)  indicates  a  breakdown  after  about  90  orbital  periods  of  propagation. 


(a)  Averaged  uncertainty  realism  test  (b)  Pearson  goodness-of-fit  test  (c)  Cramer-von  Mises  test 


Figure  8.5.4  presents  the  results  of  the  uncertainty  realism  tests  outlined  above.  All  tests  show  that  un¬ 
certainty  realism  breaks  down  the  fastest  when  the  uncertainty  is  represented  in  Cartesian  coordinates  (see 
the  red  curves).  This  is  not  surprising  since  the  non-linearity  in  the  dynamics  is  the  greatest  in  these  coordi¬ 
nates,  and  non-linearity  implies  that  the  initial  Gaussian  density  quickly  becomes  non-Gaussian.  In  contrast, 
the  osculating  equinoctial  orbital  elements  absorb  the  most  dominant  term  in  the  non-linear  dynamics  (i.e., 
the  1/r2  term  in  the  gravity)  while,  in  addition,  the  J2  equinoctial  elements  absorb  the  J2  perturbation  in 
the  gravity  [123].  As  such,  representing  uncertainty  in  one  of  these  orbital  element  systems  mitigates  the 
departure  from  “Gaussianity,”  “extends  the  life”  of  the  UKF,  and  preserves  uncertainty  realism  longer.  The 
results  inferred  from  the  averaged  uncertainty  realism  metric  in  Figure  8.5.4(a)  are  deceptive  for  the  os¬ 
culating  equinoctial  case  (green  curve)  and  suggest  that  uncertainty  realism  is  maintained  over  the  entire 
duration  of  propagation  (since  the  test  statistic  fails  to  pierce  the  99.9%  confidence  interval).  The  applica¬ 
tion  of  the  Pearson  GOF  test  in  Figure  8.5.4(b),  a  more  powerful  test  for  uncertainty  realism,  suggests  that  a 
breakdown  is  imminent  (in  the  osculating  element  case),  though  the  Pearson  test  statistic  is  still  within  the 
99.9%  confidence  interval.  On  the  other  hand,  the  Cramer-von  Mises  test  statistic,  plotted  in  Figure  8.5.4(c), 
confirms  a  potential  breakdown  in  uncertainty  realism  after  about  90  orbital  periods.  By  using  the  J2  vari¬ 
ant  of  the  equinoctial  elements  to  represent  uncertainty  (blue  curve),  the  Cramer-von  Mises  test  statistic  is 
nearly  constant  over  the  entire  duration  of  propagation  which  strongly  suggests  that  the  uncertainty  remains 
Gaussian  in  such  coordinates  and  its  uncertainty  (and  covariance)  realism  does  not  degrade.  We  also  remark 
that  the  Cramer-von  Mises  test  tends  to  produce  a  smoother  (less  noisy)  test  statistic  when  comparing  panels 
(b)  and  (c)  of  Figure  8.5.4  due  in  part  to  the  test’s  power,  robustness,  and  the  fact  that  it  is  “bin  agnostic.” 
Clearly,  this  example  shows  that  first-moment  tests  such  as  the  averaged  uncertainty  realism  test  are  not 
recommended  since  they  have  very  little  determinative  power.  Pearson’s  test  statistic  is  an  improvement,  but 
it  is  not  to  be  preferred  over  the  more  fungible  and  powerful  Cramer-von  Mises  test. 

A  more  thorough  study  of  the  test  described  above  using  a  broader  range  of  initial  conditions  is  provided 
in  the  paper  [123].  Additional  methods  for  non-linear  filtering  are  also  evaluated,  namely,  the  prediction 
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steps  of  the  extended  Kalman  and  Gauss  von  Mises  filters  [123, 126,  127],  as  are  different  coordinate  systems 
for  representing  uncertainty. 

8.5.5  Distribution  Matching  Tests  for  Orbit  Determination 

The  above  sections  on  metrics  have  focused  on  the  application  to  the  propagation  of  a  Gaussian  (covariance 
realism)  and  a  general  probability  density  function  (uncertainty  realism).  These  metrics  are  often  used  in 
several  different  contexts. 

•  For  a  single  orbit  and  real  data  with  fuzzy  truth,  one  can  compute  the  (squared)  generalized  Maha- 
lanobis  distance  U{  '  !  in  Equation  (8.3.1)  for  a  sequence  of  orbit  updates,  i.e.,  at  the  time  of  the  sensor 
report  update,  and  then  the  Cramer-von  Mises  test  statistic  (8.5.3). 

•  Again  for  a  single  orbit  but  in  a  simulation  environment,  we  can  use  multiple  Monte  Carlo  runs  with 
different  draws  of  the  sensor  reports  and  then  proceed  as  in  the  previous  example. 

•  Given  an  ensemble  of  orbits,  one  can  compute  the  (squared)  generalized  Mahalanobis  distance  Ull 
in  Equation  (8.3.1)  and  use  the  Cramer-von  Mises  test  statistic  (8.5.3)  for  the  ensemble. 

8.6  Recommendations 

This  chapter  has  defined  a  number  of  metrics  and  statistical  tests  to  assess  covariance  realism  (and  more 
general  uncertainty  realism)  for  the  propagation  of  space  object  state  uncertainty  as  well  as  orbit  determina¬ 
tion  that  additionally  enable  quantitative  comparisons  between  different  approaches.  In  addition  to  the  use 
of  the  squared  Mahalanobis  distance  (8.2.2)  and  the  corresponding  “fuzzy  truth”  version  (8.2.3),  we  have 
presented  a  generalized  Mahalanobis  distance  metric  (8.3.1)  for  use  with  more  general  probability  density 
functions.  These  can  be  used  for  the  averaged  uncertainty  realism  metric  in  Section  8.4  and  the  empirical 
distribution  matching  metric  for  uncertainty  realism  presented  in  Section  8.5.2.  These  metrics  are  intended 
for  the  aleatoric  uncertainties  that  can  be  described  in  a  probabilistic  sense.  We  have  not  addressed  epistemic 
uncertainties.  Here  are  the  recommendations. 

1 .  When  the  orbit  determination  or  uncertainty  propagation  method  produces  a  Gaussian  representation 
of  the  state  of  an  object,  we  recommend  the  averaged  Mahalanobis  test,  possibly  using  fuzzy  truth,  and 
the  Cramer-von  Mises  test  (8.5.3).  For  more  general  probability  density  functions,  we  recommend  the 
averaged,  generalized  Mahalanobis  test  (8.4.1)  and  the  Cramer-von  Mises  test  statistic  (8.5.3).  Here 
are  some  examples  of  their  use. 

•  In  uncertainty  propagation  scenarios  involving  real  data  with  truth  on  a  single  object  (e.g.,  a  high 
accuracy  orbit)  or  in  off-line  simulations  with  only  a  few  Monte-Carlo  trials  (between  1  and  10) 
in  which  there  are  insufficient  number  of  samples  to  perform  a  more  powerful  goodness-of- 
fit  (GOF)  or  distribution  matching  test,  we  recommend  using  the  averaged  uncertainty  realism 
metric.  The  use  of  such  a  metric  provides  a  necessary  condition  for  uncertainty  realism  based 
on  the  chi-squared  property  of  the  computed  test  statistic. 
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•  In  uncertainty  propagation  scenarios  involving  real  data  with  truth  on  multiple  objects  or  in 
off-line  simulations  amenable  to  a  large  number  of  Monte-Carlo  trials  (at  least  10),  the  use  of 
first-moment  tests  (such  as  the  one  based  on  the  averaged  uncertainty  realism  metric)  are  not 
recommended  due  to  their  limited  determinative  power.  In  fact,  the  application  of  such  tests  can 
lead  to  deceptive  results  as  demonstrated  in  Section  8.5.4.  Instead,  the  Cramer-von  Mises  test 
is  recommended  over  the  Pearson  GOF  test  or  the  related  Anderson-Darling  test  because  the 
former  tends  to  be  more  robust  and  less  sensitive  to  outliers  in  the  tails  of  the  distribution  and 
the  number  of  samples  used  in  the  test. 

•  For  a  single  orbit  and  real  data  with  possibly  fuzzy  truth,  one  can  compute  the  (squared)  gener¬ 
alized  Mahalanobis  distance  U^>  in  Equation  (8.3.1)  for  a  sequence  of  orbit  updates,  i.e.,  at  the 
time  of  each  sensor  report  update,  and  then  use  the  average  uncertainty  realism  test  (8.4.1)  and 
the  Cramer-von  Mises  test  statistic  (8.5.3). 

•  In  a  simulation  environment  wherein  one  can  make  multiple  Monte  Carlo  runs  with  different 
draws  of  the  sensor  reports,  one  can  compute  the  (squared)  generalized  Mahalanobis  distance 
ww  in  Equation  (8.3.1)  for  a  sequence  of  orbit  updates,  i.e.,  at  the  time  of  each  sensor  report 
update,  and  then  use  the  average  uncertainty  realism  test  (8.4.1)  and  the  Cramer-von  Mises  test 
statistic  (8.5.3)  after  each  sensor  report  update  to  the  orbit. 

•  Given  an  ensemble  of  orbits,  one  can  compute  the  (squared)  generalized  Mahalanobis  distance 
ww  in  Equation  (8.3.1)  and  use  the  average  uncertainty  realism  metric  (8.4.1)  and  the  Cramer- 
von  Mises  test  statistic  (8.5.3)  for  the  ensemble.  In  case  of  real  data,  a  covariance  measuring  the 
accuracy  of  the  data  is  needed. 

2.  Metrics  for  epistemic  uncertainties  should  be  developed. 

3.  While  the  metrics  proposed  in  this  chapter  are  applicable  for  assessing  covariance  and  uncertainty 
realism  during  propagation,  the  sensitivity  of  each  of  the  four  mission  areas  presented  in  Chapter  3  to 
a  degraded  covariance/uncertainty  realism  should  be  investigated.  Such  an  understanding  might  help 
set  some  of  the  metric  parameters  and  be  useful  in  decision  making  processes. 
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9.1  Propagation  of  Uncertainty 

The  purpose  of  this  section  is  to  define  an  initial  hierarchy  of  test  cases,  from  easy  to  challenging,  that  allow 
one  to  validate  the  performance  of  a  particular  uncertainty  propagation  method.  In  doing  so,  we  propose  a 
variety  of  states  and  covariances  representative  of  space  objects  in  different  orbital  regimes  that  are  used  to 
initialize  an  uncertainty  propagation  method  under  consideration.  Following  Horwood,  Aristoff,  Singh,  and 
Poore  [123],  the  actual  testing  is  an  off-line  process  conducted  using  multiple  Monte-Carlo  trials  and  the 
uncertainty  realism  metric  described  in  Chapter  8.  The  uncertainty  realism  metric  evaluations  collected  over 
multiple  Monte-Carlo  trials  are  then  tested  for  goodness-of-fit  using  the  averaged  uncertainty  realism  test 
or  the  Cramer- von  Mises  criterion,  both  of  which  are  defined  in  Chapter  8.  Ultimately,  the  testing  allows 
to  one  to  identify  the  first  instance  in  time  at  which  uncertainty  realism  breaks  down  during  uncertainty 
propagation  given  a  particular  set  of  initial  conditions,  filter  configuration,  and  choice  of  coordinate  system 
for  representing  uncertainty.  Full  details  of  this  testing  procedure  are  described  below. 

1.  Define  an  initial  state  and  covariance  at  time  to  by  selecting  one  of  the  scenarios  listed  in  Table  9.1.1. 

2.  Select  a  coordinate  system  in  which  to  represent  the  uncertainty,  such  as  Cartesian  ECI  or  equinoctial 
orbital  elements  (EqOE).  If  the  state  and  covariance  in  Step  1  are  not  in  this  coordinate  system,  convert 
it  to  the  targeted  coordinate  system  using,  for  example,  the  unscented  transform*. 

3.  Select  an  uncertainty  propagation  method  such  as  the  (prediction  step  of  the)  EKF,  UKF,  or  GVM 
filter.  (If  the  GVM  filter  is  selected,  then  the  Gaussian  in  Step  1  must  be  converted  to  a  GVM  distri¬ 
bution.) 

4.  Select  a  particular  (deterministic)  dynamical  model  in  the  form  of  a  system  of  ODEs  (6. 1 .2)  governing 
two-body  dynamics. 

5.  Select  propagation  times  t\, ...  ,1m-  The  final  time  Im  should  be  sufficiently  large  so  as  to  see  a 
breakdown  in  uncertainty  realism.  The  number  of  time  points  M  must  be  sufficiently  large  so  as  to 

*Even  for  the  low  accuracy  initial  covariances  in  Table  9.1.2,  this  transformation  does  not  result  in  loss  of  information,  as 
demonstrated  in  the  results  of  [123]. 
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resolve  the  time-evolution  of  the  filter  performance.  A  suggested  value  for  the  scenarios  in  Table  9.1.1 
is  M  =  200.  Note  that  sampling  only  at  multiples  of  the  orbital  period  is  discouraged. 

6.  Sample  random  particle  states,  ccW(fo),  i  =  1, . . . ,  k,  from  the  initial  PDF  defined  in  Step  1.  To 
provide  repeatability,  use  the  same  random  seed  for  each  test.) 

7.  For  each  initial  particle  state  (to),  i  =  1 , ,k,  use  an  orbital  propagator  to  generate  an  ephemeris 

x  w  (*j)  =  4>(tj  ;  (t0),  to),  for  j  =  1, ... ,  M,  where  </>  is  the  solution  flow  of  the  dynamical  model 

selected  in  Step  4. 

8.  For  j  =  1 ,M: 


(a)  Propagate  the  PDF  in  Step  1  from  time  to  to  tj  using  the  method  selected  in  Step  3  in  conjunction 
with  the  dynamical  model  specified  in  Step  4. 

(i) 

(b)  For  i  =  1,...,  /,;,  evaluate  the  uncertainty  realism  metric  U-  as  follows. 

•  If  the  propagated  PDF  in  Step  8(a)  is  represented  as  a  Gaussian,  the  uncertainty  realism 
metric  is  the  Mahalanobis  distance 


uf  =  -  PjYPj  1{xw(tj)  -  Hj), 

where  x^l\tj)  is  obtained  from  the  ephemeris  computed  in  Step  7  and  (/j,j,Pj)  is  the 
mean-covariance  pair  of  the  propagated  Gaussian  obtained  from  Step  8(a). 

•  If  the  propagated  PDF  in  Step  8(a)  is  represented  as  a  GVM  distribution,  the  uncertainty 
realism  metric  is  computed  from  Equation  (7.7.1)  where  ‘(x,  6 )’  is  given  by  x^>(tj)  (note 
that  9  is  the  last  component  in  this  ephemeris  vector)  and  ‘(/x,  P,  a,  (3,  T,  «)’  is  the  param¬ 
eter  set  of  the  propagated  GVM  distribution  obtained  from  Step  8(a). 

•  If  the  propagated  PDF  in  Step  8(a)  is  neither  a  Gaussian  nor  a  GVM  distribution,  evaluate 
the  general  uncertainty  realism  metric  specified  by  Equation  (4)  in  Chapter  8  where  ‘a;’  is 
given  by  x(l'>  (tj)  and  ‘0’  is  the  parameter  set  of  the  propagated  distribution  obtained  from 
Step  8(a). 

(i) 

(c)  Using  the  samples  U-  computed  in  Step  8(b),  compute  the  averaged  uncertainty  realism  metric 


U ,  = 


k 

1 


J  nk 

i—  1 


and  the  Cramer-von  Mises  test  statistic.  The  definition  of  the  latter  is  specified  by  Equa¬ 
tion  (8.5.3)  in  Chapter  8. 


9.  For  visualization  purposes,  plot  the  averaged  uncertainty  realism  metric  Uj  and  the  Cramer-von  Mises 
test  statistic  versus  time  tj . 

10.  Determine  the  time  when  the  averaged  uncertainty  realism  metric  Uj  and  the  Cramer-von  Mises  test 
statistic  first  pierce  a  99.9%  confidence  interval  (or  some  other  confidence  interval)  and  declare  that 
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the  uncertainty  propagation  method  has  broken  down.  Construction  of  the  confidence  intervals  used 
in  these  tests  are  provided  Chapter  8. 


Table  9.1.1  Initial  orbital  states,  with  respect  to  (osculating)  Keplerian  orbital  elements,  used  in  the  uncer¬ 
tainty  propagation  testing.  The  state  covariances  corresponding  to  the  low,  medium,  and  high  accuracy  cases 
are  listed  in  Table  9.1.2.  Depending  on  the  scenario,  the  orbit  and  its  uncertainty  are  propagated  for  1,  7,  or 
30  days,  beginning  on  1  January  2008. 


Scenario 

Orbit  Type 

Orbit  Accuracy 

a  (km) 

e 

in 

fi(°) 

w(°) 

M  (°) 

Timespan  (days) 

1 

LEO 

Low 

7136.6 

0.00949 

72.9 

116.0 

57.7 

105.5 

1 

2 

LEO 

Low 

7136.6 

0.0 

0.0 

0.0 

0.0 

0.0 

1 

3 

LEO 

High 

7136.6 

0.00949 

72.9 

116.0 

57.7 

105.5 

7 

4 

GEO 

Medium 

42164.1 

0.0 

0.0 

0.0 

0.0 

250.0 

30 

5 

HEO 

Medium 

26628.1 

0.742 

63.4 

120.0 

0.0 

144.0 

7 

This  testing  framework  is  agnostic  to  the  choice  of  perturbations  in  the  dynamical  model;  however,  we 
recommend  a  model  of  sufficient  fidelity  be  used  in  the  testing.  For  each,  in  the  paper  [123],  the  authors  used 
the  following  forces.  For  the  LEO  objects,  a  degree  and  order  32  Earth  gravity  model  was  used,  together 
with  lunar-solar  perturbations.  For  the  GEO  object,  a  degree  and  order  8  Earth  gravity  model  was  used, 
together  with  lunar-solar  perturbations  and  a  nominal  solar  radiation  pressure  ballistic  coefficient  (SRPBC) 
of  0.1  m2/kg.  For  the  HEO  object,  a  degree  and  order  32  Earth  gravity  model  was  used,  together  with 
lunar-solar  perturbations  and  a  nominal  SRPBC  of  0.1  m2/kg. 

We  remark  that  the  testing  framework  presented  here  could  be  augmented  by  providing  an  additional 
metric  that  assesses  the  computational  requirements  of  a  particular  configuration.  Although  all  of  the  filter, 
coordinate  system,  and  initial  condition  combinations  considered  in  the  study  in  Section  7.10  break  down  at 
some  future  time,  in  principle,  one  could  achieve  perfect  uncertainty  realism  indefinitely  using,  for  example, 
a  particle  filter  or  Gaussian  sum  filter  of  sufficiently  high  fidelity.  Quantifying  the  computational  demands  in 
terms  of  runtime  would  not  be  preferable  due  to  dependencies  on  computing  architectures  or  programming 
languages.  Instead,  for  sigma-point-  or  particle-based  filters  such  as  the  UKF,  GVM,  and  mixture  filters 
thereof,  we  would  recommend  quantifying  the  computational  performance  by  the  number  of  particles  (or 
sigma  points)  required  in  a  single  propagation  step' .  This  would  allow  the  user  to  evaluate  the  merit  of  a 
method  in  its  ability  to  maintain  uncertainty  realism  while  at  the  same  time  quantifying  the  computational 
cost  required  to  do  so. 


Table  9.1.2  Parameters  of  the  initial  covariances  used  in  the  uncertainty  propagation  testing.  The  orbital  state 
initial  conditions  are  listed  in  Table  9.1.1.  A  covariance  matrix  P  with  respect  to  equinoctial  orbital  elements 
is  constructed  from  a  row  in  the  table  according  to  P  =  AA7  ,  where  A  =  diag(cra,  ah,aki  ap,  aq,  at). 
Note  that  1"  =  4.848  •  10-6  radians. 


Orbit  Accuracy 

<?a  (m) 

(Jp 

aq 

Low 

20000 

10-3 

10-3 

10-3 

10-3 

36" 

Medium 

2000 

10-4 

10-4 

10-4 

10-4 

28" 

High 

50 

10-5 

10-5 

10-5 

10-5 

20" 

^This  recommendation  is  based  on  the  assumption  that  orbit  propagation  is  the  dominant  cost  in  uncertainty  propagation.  Note 
that  for  the  EKF  prediction  step,  the  dominant  cost  is  due  to  propagating  the  mean  state  vector  and  the  state  transition  matrix. 
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9.2  Recommendations 

The  test  cases  presented  in  this  chapter  are  but  an  initial  list  and  are  focused  on  the  propagation  of  uncer¬ 
tainty.  Thus,  the  recommendations  are  as  follows. 

1 .  A  list  of  test  cases  should  be  developed  to  assist  in  the  evaluation  of  each  of  the  recommendations  in 
Section  7.16.  Presumably  this  will  come  from  the  astrodynamics  community  and,  initially,  from  the 
many  publications  on  the  propagation  of  uncertainty. 

2.  Similarly,  a  set  of  test  cases  for  testing  the  correct  uncertainty  at  epoch  should  be  developed. 
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10.1  The  Problem  Addressed 

The  purpose  of  this  report  has  been  to  delineate  the  many,  but  not  all,  issues  that  must  be  addressed  if  one 
is  to  achieve  a  correct  characterization  of  the  uncertainty  in  the  estimated  state  (e.g.,  position  and  velocity) 
of  a  space  object.  As  stressed  in  Chapter  3,  this  characterization  is  fundamental  to  achieving  robust  SSA 
functions  such  as  conjunction  assessments,  data  association,  maneuver  detections,  and  sensor  resource  man¬ 
agement  in  the  anticipated  space  environment  of  the  future  wherein  the  automation  of  these  processes  will 
be  required.  While  this  report  surveys  many  candidate  algorithms  for  achieving  this  characterization,  the 
ranking  of  these  algorithms  has  been  specifically  avoided  in  preference  for  a  set  of  benchmark  test  cases  and 
metrics  to  evaluate  such  algorithms.  Only  in  this  way,  can  a  reasonably  unbiased  conclusion  be  made. 

10.2  Generic  Uncertainties 

Covariance  and  uncertainty  realism  is  the  proper  characterization  of  the  “uncertainty”  in  the  state  of  an 
object.  Uncertainty  realism  as  defined  in  Section  2.1  is  limited  to  aleatoric  anchorites  and  is  that  of  having  a 
correct  probability  density  function  (PDF)  that  characterizes  the  “uncertainty”  and  “errors”  in  the  state  of  an 
object.  As  such,  it  generalizes  covariance  uncertainty  which  only  requires  the  correct  mean  and  covariance 
matrix,  but  not  necessarily  the  higher  order  moments.  In  the  remainder  of  this  chapter,  the  term  uncertainty 
realism  will  be  used  to  include  both  cases  of  a  general  PDF  and  a  Gaussian  PDF. 

The  general  subject  of  uncertainty  realism  is  one  that  is  currently  an  active  area  of  research  in  the  field 
of  “uncertainty  quantification”  in  many  areas  of  science  and  engineering  and  one  that  is  briefly  outlined  in 
Chapter  2  with  references  to  some  of  the  extensive  literature.  A  recommendation  (Section  2.2)  then  is  to 
keep  abreast  of  the  developments  in  this  field.  The  two  types  of  uncertainties  treated  in  this  field  are  called 
aleatoric  and  epistemic  uncertainties.  This  report  addresses  aleatoric  uncertainties,  i.e.,  those  that  can  be 
characterized  probabilistically.  Thus,  a  recommendation  is  to  investigate  the  use  of  epistemic  or  a  mixture 
of  both  aleatoric  and  epistemic  in  treating  uncertainties  in  astrodynamics. 

The  general  goal  of  uncertainty  realism  for  space  surveillance  is  that  of  assessing  and  rolling  up  the  dif¬ 
ferent  “uncertainties”  and  “errors”  into  the  state  of  an  object  at  epoch,  i.e.,  the  time  of  the  last  measurement, 
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and  then  propagating  this  uncertainty  to  some  future  time  in  support  of  SSA  missions,  e.g.,  conjunction  as¬ 
sessments,  data  association,  maneuver  detection,  and  sensor  resource  management  discussed  in  Chapter  5. 
Such  an  effort  starts  with  a  characterization  the  parametric  and  structural  uncertainties  in  the  astrodynamics 
equations  of  motion  in  Chapter  4  including  the  uncertainties  in  the  space  environment.  Next,  sensor  level 
processing  produces  the  sensor  reports  used  in  initial  or  orbit  determination.  The  estimation  of  the  state 
in  the  presence  of  possibly  unrealistic  uncertainties  in  the  sensor  reports  is  the  next  step,  followed  by  a 
propagation  of  the  state  and  its  uncertainty  to  a  future  time  as  required  by  some  SSA  missions. 

The  specific  recommendations  in  the  report  are  contained  in  the  Sections  2.2,  4.3,  5.8,  6.12,  7.16,  8.6, 
and  9.2.  Rather  than  repeating  these  recommendations  in  the  chapter-by-chapter  order,  it  may  be  instruc¬ 
tive  to  briefly  re-examine  these  recommendations  from  the  viewpoint  of  the  generic  sources  of  uncertainty 
presented  in  Section  1.1. 

10.2.1  Structural  Uncertainties 

There  are  two  basic  approaches  to  the  treatment  of  structural  uncertainty  in  the  astrodynamics  equations 
of  motion,  namely  the  use  of  process  noise  model  such  as  those  presented  in  Section  6.1.1  and  an  optimal 
control  approach  presented  in  Sections  3.3,  6.2,  6.6,  and  6.7.  (The  use  of  a  controls  approach  assumes  that 
the  data  association  is  correct;  otherwise,  one  might  tit  the  dynamics  to  two  different  objects.)  These  appear 
to  be  a  fruitful  approaches,  especially  if  the  two  views  can  augment  each  other.  Astrodynamics  models  and 
an  enumerated  list  of  uncertainties  associated  with  the  equations  of  motion  are  presented  in  Chapter  4  with 
the  recommendations  in  Section  4.3. 

10.2.2  Uncertain  Parameters 

Parameters  are  present  in  the  dynamics  and  measurement  equations.  Many  of  the  parameters  in  the  dynamics 
are  reviewed  in  Chapter  4  and  methods  for  treating  them  within  the  obit  determination  process  are  presented 
in  Section  6.2.2.  (The  most  dominant  uncertainties  are  those  of  solar  radiation  pressure  and  atmospheric 
drag;  a  more  complete  list  can  be  found  in  Section  4.3.)  Some  parameters  may  be  estimated,  but  others 
may  not  due  to  observability  issues.  In  the  former  case,  the  parameters  are  estimated  as  part  of  the  orbit 
determination  procedure,  and  the  uncertainty  in  the  estimate  is  incorporated  as  a  consider  parameter.  In  the 
latter  case,  one  generally  should  consider  the  methods  of  epistemic  uncertainties  to  represent  the  uncertainty, 
but  aleatoric  methods  are  frequently  used  as  discussed  in  Section  6.2.2.  In  addition,  there  are  parameters 
in  the  predicted  atmospheric  density  due  to  a  dynamic  space  weather  environment  that  are  required  for 
predicting  the  state  and  its  uncertainty  into  the  future  from  the  last  sensor  report,  i.e.,  epoch.  (While  extensive 
efforts  have  been  extended  to  the  development  of  algorithms  for  propagating  structural,  parametric,  and 
initial  state  uncertainties  forward  in  time,  having  a  statistically  realistic  model  for  the  predicted  impacts  of 
space  weather  is  essential  to  achieving  realistic  uncertainties  in  the  predicted  state  for  near-earth  objects.) 
The  objective  in  dealing  with  uncertain  parameters  is  their  estimation  and  ranking  of  the  uncertainties  so  that 
at  least  the  most  dominant  ones  are  treated.  There  is  much  outstanding  research  in  the  treatment  of  these 
topics  including  the  use  of  a  consider  or  Schmidt-Kalman  filter  to  better  account  for  the  statistical  cross 
correlation  between  the  state  and  the  uncertain  parameters,  thereby  improving  the  uncertainty  in  the  state. 
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In  the  measurement  equation  that  relates  the  state  to  the  sensor  report,  the  uncertain  parameters  are 
generally  delineated  as  sensor  biases,  navigation  errors,  and  time  biases.  These  are  normally  treated  as  part 
of  the  calibration  process  as  discussed  in  the  next  subsection  and  more  extensively  in  Chapter  5. 

10.2.3  Sensor  Measurement  Noise 

The  process  of  generating  sensor  reports  is  a  complicated  one  that  could  not  be  fully  investigated  in  this 
report  due  to  both  time  and  the  availability  of  documents  at  the  Distro-A  level;  however,  improved  char¬ 
acterizations  of  the  sensor  reports  and  their  uncertainty  may  well  be  the  biggest  contributor  to  improved 
covariance  and  uncertainty  realism.  For  example,  a  radar  may  generate  reports  in  the  following  way.  A 
space  object  may  be  detected  in  the  natural  intrinsics  of  the  radar,  e.g.  (r,  u,  v )  measurements,  which  are 
then  filtered.  The  filtered  state  (and  hopefully  its  covariance)  is  then  transformed  to  a  different  measurement 
space  such  as  range,  azimuth,  and  elevation.  Such  sensor  reports  arc  thus  highly  correlated  and  the  covari¬ 
ances  unrealistic.  Compounding  this  problem  is  that  of  the  different  biases  including  atmospheric,  sensor, 
navigation,  and  time  which  can  vary  with  time.  Ideally,  one  would  report  the  different  biases,  their  uncertain¬ 
ties,  the  sensor  measurement  and  its  uncertainties  in  the  intrinsic  measurement  space.  This  would  potentially 
offer  the  best  opportunity  for  treating  uncertainty  at  the  sensor  level.  Thus,  a  recommendation  is  to  undertake 
a  program  to  investigate  the  potential  improvement  in  the  achievement  of  covariance/uncertainty  realism  at 
the  sensor  level  by  better  processing.  Much  more  extensive  justification  for  this  recommendation  can  be 
found  in  Chapter  5  and,  in  particular,  the  short-term  to  long-term  recommendations  in  Section  5.8. 

10.2.4  Inverse  Uncertainty  Quantification 

The  inverse  problem  includes  the  statistical  orbit  determination  and  bias  estimation.  The  estimation  of  the 
state  of  an  object  and  its  uncertainty  given  the  sensor  reports  and  astrodynamics  models  is  critical  to  the 
propagation  of  uncertainty.  Also  included  in  this  subject  is  the  estimation  of  the  sensor,  navigation,  or  time 
biases  given  truth  objects  or  fuzzy  truth  objects  (e.g.,  high  accuracy  orbits)  and  the  statistical  characterization 
of  the  uncertainty  in  these  estimates.  (An  inverse  problem  is  a  general  framework  that  is  used  to  convert 
observed  measurements  into  information  about  a  physical  object  or  system.  A  bias  is  a  systematic  error  that 
does  not  average  out.)  A  brief  synopsis  of  the  estimation  techniques  used  in  astrodynamics  was  presented 
in  Chapter  6  and  includes  the  estimation  of  the  state  of  an  object  as  well  as  the  sensor,  navigation,  time, 
and  atmospheric  biases  in  sensor  level  processing.  Of  particular  note,  is  the  use  of  empirical  covariances  as 
opposed  to  the  theoretical  covariances  as  described  in  the  recommendations  in  Chapter  6. 

10.2.5  Propagation  of  uncertainty 

The  propagation  of  uncertainty  in  state  of  an  object  and  its  uncertainty  through  nonlinear  dynamics  or 
to  transformation  through  nonlinear  functions  changes  the  form  of  the  uncertainty  characterization.  For 
example,  initially  characterized  as  a  Gaussian  will  become  non-Gaussian.  Included  in  this  problem  is  that 
of  predicting  the  space  weather  environment  in  the  future. 

Coordinate  systems  discussed  briefly  in  7.2  are  particularly  important  in  the  length  of  time  uncertainty 
can  be  maintained  by  a  particular  filtering  method.  For  example,  element  spaces  absorb  the  dominant  term 
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in  the  gravity  and  thereby  make  the  representation  of  the  dynamics  less  nonlinear.  At  least  one  complete 
set  of  algorithms  have  been  developed  that  start  with  uncertainty  represented  in  either  Cartesian  space  or 
element  space,  represents  the  uncertainty  in  element  space,  propagates  sigma  points  or  particles  in  the  iner¬ 
tial  Cartesian  space,  and  then  transforms  the  sigma  points  (or  particles)  back  to  element  space  wherein  the 
uncertainty  can  be  recovered.  Next  if  the  uncertainty  is  required  in  another  coordinate  system,  a  Gaussian 
sum  representation  of  the  uncertainty  can  be  used  to  transform  the  uncertainty  7.9.  Alternately,  one  can 
simply  transform  the  uncertainty  in  a  Cartesian  space  using  an  adaptive  Gaussian  sum  representation. 

As  noted  in  Chapter  7  this  area  of  uncertainty  quantification  has  been  extensively  developed  with  virtu¬ 
ally  all  methods  of  estimation  applied  to  this  problem  in  astrodynamics.  One  can  consider  this  as  a  maturing 
subject.  What  is  missing  are  papers  on  the  comparison  of  the  different  methods. 

While  this  report  does  not  compare  all  the  methods  presented  above,  the  computational  results  in  Section 
7.10  indicate  the  performance  advantage  of  the  representation  of  uncertainty  in  orbital  element  space  and 
the  performance  of  the  UKF,  GVM  filter,  and  Gaussian  sums  in  element  space.  A  more  detailed  study 
that  uses  a  common  set  of  metrics  and  test  cases,  such  as  those  described  in  Section  9.1,  and  applies  them 
to  the  uncertainty  propagation  methods  reviewed  in  this  chapter  should  be  undertaken  in  the  future.  Such 
comparisons  need  to  be  extended  to  the  vast  number  of  methods  presented  in  Chapter  7.  To  better  understand 
the  strengths  (and  weaknesses)  of  the  above  methods,  one  needs  a  set  of  test  problems,  a  set  of  performance 
metrics,  and  a  list  of  test  cases  that  can  help  evaluate  these  strengths  and  weaknesses.  While  performance 
metrics  are  discussed  in  the  next  section,  here  is  a  list  of  potential  comparisons. 

•  An  assessment  of  the  range  of  validity  for  each  method  for  propagating  uncertainty, 

•  Time  the  representation  stays  within  a  specified  confidence  region, 

•  Performance  with  and  without  process  noise  or  stochastic  acceleration, 

•  Performance  with  and  without  model  mismatch, 

•  Performance  as  a  function  of  the  size  of  the  initial  uncertainty, 

•  Computational  complexity, 

•  Dependence  on  the  different  regimes  of  space. 

In  addition  to  propagation  of  uncertainty  forward  in  time,  the  transformation  of  uncertainty  from  one  coor¬ 
dinate  system  to  another  should  be  considered. 

The  newer  orbital  propagators  based  on  implicit  Runge-Kutta  methods  or  modified  Picard  iteration  as 
presented  in  Section  7.15  are  recommended  due  to  their  numerical  stability  properties,  efficiency  due  to 
large  step  sizes,  adaptability  to  high  performance  computing  and  parallelization,  and  the  ability  to  propagate 
an  ensemble  of  orbits  needed  in  the  unscented  Kalman  filter,  Gauss-Hermite  filters,  Gauss  von  Mises  filter, 
or  particle  filters. 

10.2.6  Algorithmic  Uncertainty 

Algorithmic  or  numerical  uncertainty  comes  from  numerical  errors  and  numerical  approximations  in  a  com¬ 
puter  model.  An  example  is  the  truncation  error  in  the  orbital  propagator,  which  can  be  mitigated  by 
adjusting  the  numerical  error  tolerance  so  that  the  magnitude  of  the  numerical  error  to  ensure  that  it  is  below 
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that  of  the  uncertainty  due  to  the  physics  or  uncertainty  in  the  state  of  an  object.  Another  example  is  that  of 
orbit  determination  in  the  presence  of  ill-conditioning. 

10.2.7  Cross-tag  or  Misassociation  Uncertainty 

Misassociation  uncertainty  is  discrete  in  nature  and  is  generally  characterized  through  the  computation  of 
the  probability  of  association  [166,  205].  Cross-tags  can  degrade  the  uncertainty  in  the  state  of  an  object 
and  is  especially  important  for  closely  spaced  objects  such  as  GEO  clusters,  LEO  breakups,  and  tethered 
satellites. 


10.2.8  Hardware  and  Software  Faults/Srrors 

These  non-algorithm  errors  are  yet  another  source  of  uncertainty  introduced  into  the  process,  but  arc  not 
addressed  in  this  report. 

10.2.9  Extended  Body  Uncertainties 

Extended  body  uncertainties  represent  an  additional  source  of  uncertainty  for  medium  to  large  objects.  For 
example,  an  extended  body  covering  several  pixels  may  have  an  overly  optimistic  (too  small)  covariance  if 
the  uncertainty  of  the  estimated  state  only  covers  the  centroid  of  the  body.  One  must  also  address  the  extent 
of  the  body,  possibly  through  the  use  of  feature  data  such  as  multi-band  photometry  and  rad  ionic  try.  Radar, 
on  the  other  hand,  may  receive  reflections  from  multiple  point  scatterers  and  the  centroided  uncertainty  may 
not  cover  the  extent  of  the  body.  (We  note  that  the  radar  cross  section  (RCS)  of  an  extended  object  is  not 
well  defined.  Each  point  scatterer  may  have  a  different  RCS.  Thus  the  “object  RCS”  could  be  the  mean  or 
the  maximum  of  the  point  scatterers.) 

10.3  Metrics 

Tests  or  metrics  for  evaluating  covariance  and  uncertainty  realism  are  an  essential  part  of  evaluating  the 
different  methods  for  generating  uncertainty  of  the  propagated  state  or  uncertainty  at  epoch.  The  focus 
of  the  metrics  in  this  report  has  been  that  of  the  propagated  state  and  should  be  extended  to  that  of  the 
uncertainty  at  epoch.  In  general,  when  the  orbit  determination  or  uncertainty  propagation  method  produces 
a  Gaussian  representation  of  the  state  of  an  object,  we  recommend  the  averaged  Mahalanobis  test,  possibly 
using  fuzzy  truth,  and  the  Cramer-von  Mises  test  (8.5.3).  For  more  general  probability  density  functions,  we 
recommend  the  averaged,  generalized  Mahalanobis  test  (8.4.1)  test  and  the  Cramer-von  Mises  test  statistic 
(8.5.3)  as  discussed  in  Chapter  8. 

In  future  work,  metrics  for  evaluating  uncertainty  at  epoch  will  be  required.  Furthermore,  metrics  for 
epistemic  uncertainties  should  be  developed.  While  the  above  addresses  uncertainty  realism,  the  sensitivity 
of  each  of  the  four  mission  areas  presented  in  Chapter  3  to  a  degraded  uncertainty  realism  should  be  inves¬ 
tigated  in  conjunction  with  the  above  metrics..  Such  an  understanding  might  help  set  some  of  the  metrics 
parameters  and  be  useful  in  decision  making  processes. 
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Metrics  for  evaluating  uncertainty  at  epoch,  i.e.,  the  time  of  the  last  sensor  report  should  be  undertaken; 
however,  this  development  is  probably  best  made  in  conjunction  with  improving  uncertainty  realism  in  the 
sensor  reports. 

10.4  Some  Specific  Topics  Not  Addressed 

While  this  report  addresses  many  of  the  above  sources  of  uncertainty,  it  does  not  address  all  of  those  sources 
listed  listed  in  Section  1.1.  Here  is  such  a  list  plus  additional  metrics. 

•  While  the  subject  of  algorithms  is  briefly  discussed  throughout  the  different  sections,  there  is  no  in- 
depth  treatment  of  such  algorithms.  In  particular,  there  is  no  comparison  between  filter/smoothing 
and  batch  processing  and  the  corresponding  algorithms. 

•  No  attempt  is  made  to  address  extended  body  uncertainties. 

•  All  of  the  uncertainties  and  errors  treated  in  this  report  are  of  the  aleatoric  type.  The  subject  of 
epistemic  and  mixtures  of  epistemic  and  aleatoric  uncertainties  should  be  investigated  and  addressed 
in  future  work.  In  addition,  metrics  for  epistemic  uncertainties  will  be  needed. 

•  No  attempt  is  made  to  address  cross-tag  or  misassociation  uncertainty,  which  is  treated  to  some  extent 
in  Drummond  [77]. 

•  Metrics  and  test  cases  for  the  estimate  of  the  state  and  its  uncertainties  at  epoch  have  not  been  specif¬ 
ically  addressed,  but  should  be  in  future  work. 

•  Most  of  the  recommendations  require  further  investigations,  tests  and  evaluations  for  potential  op¬ 
tional  approaches.  In  addition,  the  report  does  not  provide  an  estimate  of  the  expected  total  level  of 
effort  in  completing  the  goals  and  neither  addressing  potential  issues  or  impact  on  system  interoper¬ 
ability. 


10.5  Next  Steps 

The  report  offered  an  analytical  survey  on  covariance  and  uncertainty  realism  in  space  surveillance  and  a 
list  of  recommendations  in  each  technical  area  for  further  evaluation  of  various  approaches.  It  does  not  give 
a  comprehensive  review  in  addressing  and  identifying  the  current  (or  legacy)  operational  system  and  its  de¬ 
ficiencies  with  respect  to  the  expected  requirements  (current  or  future)  to  justify  the  R&D  recommendations 
listed  in  the  report.  This  is  due  in  part  to  the  Distro-A  requirement.  In  keeping  with  this  requirement,  the 
audience  for  this  report  is  a  general  one. 

The  recommendations  at  the  end  of  each  chapter  are  focused  on  the  specific  topics  in  that  chapter.  Some 
areas,  such  as  the  propagation  of  uncertainty  discussed  in  Chapter  7,  have  received  much  attention  in  the 
astrodynamics  community  and  is  now  maturing,  while  other  fundamentally  important  areas  such  as  error 
characterization  of  the  sensor  reports  in  Chapter  5  have  received  little  attention.  Hence,  this  final  section  is 
designed  to  outline  what  the  next  steps  might  be. 
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A.  The  first  recommended  “next  step”  is  that  a  companion  study  be  undertaken  to  examine  existing  system 

capabilities  and  known  deficiencies  in  order  to  establish  the  basis  for  undertaking  a  research  and 
development  program  and  analysis  effort.  Recommendations  coming  out  of  this  study  should  then  be 
made  for  short,  medium,  and  long  term  improvements. 

B.  As  noted  in  Chapter  5,  one  of  the  most  critical  difficulties  for  operational  SSA  and  space  surveillance  is 

that  the  input  sensor  reports  can  be  of  very  limited  quantity,  quality,  and  poor  distribution.  In  addition, 
there  can  be  huge  uncertainties  in  the  weather  forecast  at  near-earth  altitudes.  Yet,  there  are  few 
publications  on  these  topics.  Without  key  improvements  in  the  error  characterization  of  input  data 
and  weather  forecasting,  most  of  the  research  into  other  aspects  of  uncertainty  quantification  in  SSA 
(e.g.,  propagation  of  uncertainty)  may  not  give  the  implied  improvement  in  the  overall  operational 
system.  An  extensive  set  of  recommendation  is  provided  in  Section  5.8  for  sensor  level  processing. 
From  a  research  and  development  viewpoint,  attention  to  improvements  in  sensor  level  processing  and 
space  weather  prediction  models  is  needed  before  one  can  take  full  advantage  of  the  ongoing  research 
in  uncertainty  quantification  of  the  orbits. 

C.  The  Committee  recommends  the  development  of  a  repository  of  benchmark  test  problems,  metrics  to 

evaluate  performance,  and  test  results  so  that  the  community  can  make  assessments  of  the  developed 
or  proposed  algorithms.  Since  the  topic  of  propagation  of  uncertainty  appears  to  be  maturing,  one 
could  start  with  this  topic  assuming  the  covariance  or  PDF  at  epoch  is  realistic. 

D.  This  report  has  proposed  new  metrics  for  the  evaluation  of  uncertainty  realism  in  the  propagated  state 

of  an  object,  however,  the  real  test  should  be  that  of  degraded  performance  in  the  aforementioned 
mission  areas  in  Chapter  3.  While  some  of  the  sensitivity  of  these  mission  to  unrealistic  covariances 
has  been  examined  in  the  literature,  the  focus  of  metrics  must  eventually  be  that  of  the  impact  on  the 
mission  areas.  Likewise,  the  sensitivity  to  unrealistic  input  data  should  be  undertaken. 

E.  The  development  of  improved  dynamic  models  is  one  method  of  reducing  uncertainty  in  the  state  of  an 

object  and  such  improvements  are  certainly  to  be  continued.  The  emphasis  on  the  use  of  methods  of 
“uncertainty  quantification”  represents  a  shift  in  the  approach  to  estimating  the  state  and  its  uncertainty 
in  a  given  model.  The  goal  here  then  is  to  take  a  given  model,  to  quantify  its  uncertainty,  and  to  use 
the  uncertainty  in  support  of  the  mission  areas  discussed  in  Chapter  3.  The  use  of  optimal  control 
and  process  noise  still  offer  an  opportunity  to  account  for  model  mismatch  and  has  not  yet  been  fully 
exploited. 

F.  Chapter  4  orders  the  different  parameters  in  the  dynamics  and  space  weather  in  order  of  their-  uncertain¬ 

ties.  A  more  extensive  survey  of  the  literature  should  be  undertaken  with  recommended  directions  of 
development.  It  is  anticipated  that  a  future  report  on  New  Methods  of  Reentry  Modeling  will  consid¬ 
erably  expand  the  topic  of  uncertainty  quantification  of  reentering  objects. 

G.  A  conference  or  special  session  in  one  of  the  astrodynamics  conferences  should  be  scheduled  to  discuss 

the  pressing  problem  of  achieving  uncertainty  realism  in  SSA. 
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The  following  provides  a  glossary  for  symbols,  abbreviations,  and  acronyms. 

AFRL  Air  Force  Research  Laboratory 
ASAT  anti-satellite 

CBTA  covariance  based  track  association 

CCD  charge-coupled  device 

CDF  cumulative  distribution  function 

CIO  celestial  intermediate  origin 

DA  differential  algebra 

DC  differential  correction 

DCP  dynamic  consider  parameter 

DMC  dynamic  model  compensation 

ECDF  empirical  cumulative  distribution  function 

ECI  Earth  centered  inertial 

EDF  empirical  distribution  function 

EKF  extended  Kalman  filter 

EO  electro-optical 

EOP  Earth  orientation  parameters 

EUV  extreme  ultraviolet 

FISST  finite  set  statistics 

FPKE  Fokker-Planck-Kolmogorov  equation 

GCRF  geocentric  celestial  reference  frame 

GEO  geostationary  orbit 

GOF  goodness  of  fit 

GMST  Greenwich  mean  sidereal  time 

GVM  Gauss  von  Mises 

HASDM  high  accuracy  satellite  drag  model 

HBR  hard  body  radius 

HEO  highly  elliptical  orbit 

IAU  International  Astronautical  Union 

IERS  International  Earth  Rotation  and  Reference  Systems  Service 


125 


10.5.  NEXT  STEPS 


CHAPTER  10.  CONCLUSIONS 


ITRF  International  terrestrial  reference  frame 

IOD  initial  orbit  determination 

IMM  interacting  multiple  model 

JPDA  joint  probabilistic  data  association 

JSC  Johnson  Space  Center 

JSpOC  Joint  Space  Operations  Center 

LEO  low  Earth  orbit 

MAP  maximum  a  posteriori 

MEO  medium  Earth  orbit 

MFA  Multiple  Frame  Assignments 

MHT  Multiple  Hypothesis  Tracking 

ML  maximum  likelihood 

MMSE  Minimum  Mean  Squared  Error 

NASA  National  Aeronautics  and  Space  Administration 

NASA/GSFC  NASA  Goddard  Space  Flight  Center 

OD  orbit  determination 

PC  polynomial  chaos 

Pc  probability  of  collision 

PDF  Probability  Density  Function 

PMF  Probability  Mass  Function 

R.0  radius  of  the  Earth 

RSO  resident  space  object 

RMS  root  mean  square 

SDE  stochastic  differential  equation 

SOEKF  second  order  extended  Kalman  filter 

SP  Special  Perturbations 

SSA  space  situational  awareness  The  requisite  current  and  predictive  knowledge  of  the  space  environment 
and  the  operational  environment  upon  which  space  operations  depend  as  well  as  all  factors,  activities, 
and  events  of  friendly  and  adversary  space  forces  across  the  spectrum  of  conflict. 

SRP  solar  radiation  pressure 

SSN  space  surveillance  network 

STSS  space  tracking  surveillance  system 

UKF  unscented  Kalman  filter 

QMU  quantification  of  margins  and  uncertainties 

TAI  temps  atomique  international  (international  atomic  time) 

TCA  time  of  closest  approach 
TT  terrestrial  time 
UQ  uncertainty  quantification 
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UTC  universal  time  coordinated 
V&y  verification  and  validation 
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